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PARTNERSHIP     FOR     ADVANCED     COMPUTA- 
TIONAL INFRASTRUCTURE  PROGRAM 


TUESDAY,  MARCH  19,  1996 

U.S.  House  of  Representatives, 

Committee  on  Science, 
Subcommittee  on  Basic  Research, 

Washington,  DC. 

The  Subcommittee  met  at  1  p.m.  in  Room  2318  of  the  Rayburn 
House  Office  Building,  the  Honorable  Steven  H.  Schiff,  Chairman 
of  the  Subcommittee,  presiding. 

Mr.  Schiff.  The  Subcommittee  will  please  come  to  order. 

Today  the  Subcommittee  will  exercise  its  oversight  responsibil- 
ities for  the  National  Science  Foundation  by  receiving  testimony  on 
the  solicitation  of  the  Partnership  for  Advanced  Computational  In- 
frastructure Program. 

As  this  Congress  continues  to  find  new  ways  to  balance  the  budg- 
et and  reduce  the  size  of  government,  agencies  are  looking  to  maxi- 
mize the  value  of  each  taxpayer  dollar. 

Of  the  government  agencies,  the  National  Science  Foundation  is 
one  of  the  best  Federal  agencies  at  running  a  lean  and  efficient  or- 
ganization. I  extend  my  compliments  to  Dr.  Lane  and  all  of  his  em- 
ployees. 

This  Congress  provided  the  NSF  with  an  increase  in  FY  1996 
funding  over  FY  1995,  but  hard  choices  must  still  be  made  on  fund- 
ing priorities  and  the  direction  programs  should  be  taking  as  we 
head  into  the  21st  Century. 

One  of  those  programs  undergoing  review  is  NSF's 
Supercomputer  Center  program.  NSF  is  making  tough  decisions  on 
the  direction  of  the  Supercomputer  Centers,  their  funding,  and 
what  role  they  should  play  in  research  applications. 

The  Partnership  for  Advanced  Computational  Infrastructure  Pro- 
gram builds  on  and  replaces  the  current  NSF  Supercomputer  Cen- 
ters program  established  in  1985.  The  new  program  will  focus  on 
taking  advantage  of  newly  emerging  opportunities  in  high-perform- 
ance computing  and  communications. 

The  Subcommittee  is  interested  in  assessing  the  major  accom- 
plishments of  the  NSF's  Supercomputing  Centers  program  over  the 
last  10  years,  as  well  as  the  proposed  restructuring  of  the  program. 

Additionally,  the  subcommittee  desires  to  receive  any  comments 
or  specific  suggestions  from  witnesses  on  the  recompetition  process 
which  NSF  has  developed  for  the  program. 

To  outline  a  brief  history  of  the  Supercomputer  program,  the 
NSF  Supercomputing  Centers  Program  was  established  in  1984  fol- 
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lowing  strong  expressions  of  the  need  for  such  computing  resources 
for  the  academic  research  community. 

During  a  review  in  1990,  there  was  a  distinct  effort  to  expand 
outreach  services  with  initial  efforts  intended  to  forge  closer  ties  to 
industries  that  could  profit  from  exposure  to  high-performance 
computing  and  to  include  the  community  at  large. 

The  Task  Force  Report  we  will  receive  testimony  on  today  is  the 
latest  of  many  studies  of  the  Centers.  The  Director  of  the  National 
Science  Foundation  established  the  Task  Force  on  the  Future  of 
the  NSF  Supercomputer  Centers  Program  in  December  1994. 

The  Task  Force  was  asked  to  analyze  various  alternatives  for  the 
continuation,  restructuring,  or  phaseout  of  NSF's  current 
Supercomputing  Centers  Program  or  the  development  of  similar  fu- 
ture programs,  and  to  make  recommendations  among  the  alter- 
natives. 

In  January  the  National  Science  Board  adopted  the  recommenda- 
tion for  a  new  competition  of  the  Centers,  and  NSF  announced  the 
Partnership  for  Advanced  Computational  Infrastructure. 

Preliminary  solicitations  are  due  April  15th,  1996.  The  final 
awards  will  be  announced  in  the  fall  of  1996. 

To  learn  more  about  the  partnership  program,  it  is  a  pleasure  to 
welcome  Dr.  Paul  Young,  the  Assistant  Director  for  Computer  and 
Information  Science  and  Engineering  at  the  National  Science  Foun- 
dation. 

We  also  welcome  Dr.  Ed  Hayes,  Chairman  of  the  Task  Force  on 
the  Future  of  NSF  Supercomputing  Centers  Program  who  rec- 
ommended the  National  Science  Board  have  a  recompetition  of  the 
existing  Supercomputer  Centers. 

For  our  second  panel  we  will  welcome  the  directors  of  the  four 
Centers.  Dr.  Malvin  Kalos  of  the  Cornell  Theory  Center;  Dr.  Larry 
Smarr  of  the  National  Center  for  Computing  Applications;  Dr. 
Douglas  Pewitt  of  the  San  Diego  Supercomputing  Center;  and  from 
the  Pittsburgh  Supercomputing  Center  we  have  Dr.  Michael  Levine 
and  Dr.  Ralph  Roskies. 

On  our  third  panel  we  will  hear  from  supercomputer  researchers 
Dr.  Mary  Vernon  and  Dr.  Kelvin  Droegemeir.  We  will  hear  also 
from  Dr.  Douglas  Gale  who  will  present  the  views  of  organizations 
which  want  to  be  involved  in  the  new  supercomputing  program  at 
NSF. 

Before  calling  the  first  panel  up  to  testify,  I  first  want  to  wel- 
come officially  here  at  the  Subcommittee — I  do  not  know  that  we 
have  done  it  on  the  record  in  the  Full  Committee  yet — but  welcome 
officially  to  the  Subcommittee  my  new  Ranking  Member  from  Ala- 
bama, Congressman  Cramer,  and  you  are  recognized  for  whatever 
remarks  you  might  like  to  make  at  this  time. 

Mr.  Cramer.  Thank  you,  Mr.  Chairman.  I  will  look  forward  to 
working  with  you  and  the  Committee  Members  both  on  my  side  of 
the  aisle  and  your  side  of  the  aisle,  as  well. 

I  am  pleased  to  join  the  chairman  in  welcoming  our  distin- 
guished witnesses  this  afternoon.  I  congratulate  him  on  calling  the 
hearing  on  this  important  subject  matter,  the  Future  of  the  Na- 
tional Science  Foundation's  Supercomputer  Centers. 

Today  we  are  reviewing  an  NSF  program  which  this  subcommit- 
tee has  supported  and  encouraged  since  its  inception  ten  years  ago. 


Hearings  in  the  early  1980s  highlighted  concerns  from  the  aca- 
demic research  community  that  researchers  had  very  limited  access 
to  advanced  scientific  computers.  This  was  of  concern  because  high- 
performance  computers  were  beginning  to  show  enormous  promise 
as  tools  for  attacking  previously  intractable  problems.  They  had 
opened  entirely  new  areas  of  scientific  inquiry. 

At  the  outset,  the  Supercomputer  Centers  Program  simply  pro- 
vided computer  time  to  academic  researchers  at  a  few  existing 
supercomputer  sites.  That  is  not  the  way  it  works  anymore. 

The  program  was  subsequently  enlarged  in  scope  to  accelerate 
the  development  and  use  of  new  hardware  and  software  for  science 
and  engineering  applications,  and  to  expand  the  number  of  re- 
searchers skilled  in  the  use  of  advanced  computing  technologies. 

The  NSF  Supercomputer  Centers  have  now  become  important  in- 
tellectual centers  driving  the  rapid  progress  of  scientific  computing. 

The  impact  of  this  program  is  summed  up  very  well  in  a  recent 
report  of  the  Task  Force  on  the  Future  of  the  NSF  Supercomputer 
Centers  Program. 

Thorough  review  and  assessment  of  the  Supercomputer  Centers 
Program  after  10  years  is  both  reasonable  and  expected.  NSF  has 
instituted  two  major  external  reviews  over  the  past  four  years  pro- 
viding many  opportunities  for  the  affected  research  community  to 
express  their  views. 

The  restructuring  plan  for  the  Centers  program,  which  the  sub- 
committee will  hear  more  about  today,  has  resulted  from  the  rec- 
ommendations of  the  Hayes  Task  Force.  I  am  pleased  that  Dr. 
Hayes  is  present  to  discuss  them. 

I  would  like  to  call  attention  to  one  particular  recommendation 
of  this  Hayes  Task  Force,  which  was.  That  the  Leading  Edge  Com- 
puter Center  should  be  partnered  with  Regional  Computer  Centers 
for  maximum  benefit  to  the  academic  research  community. 

We  have  such  a  regional  computer  center  in  my  Congressional 
District  in  Huntsville,  Alabama,  and  I  would  like  to  submit  for  the 
record  a  written  statement  from  the  Alabama  Supercomputer  Au- 
thority, which  indicates  some  of  the  activities  of  the  Center  and  in- 
dicates how  regional  centers  can  contribute  to  the  new  NSF  Part- 
nerships Program. 

In  light  of  the  past  performance  of  the  Supercomputer  Centers, 
any  proposal  for  restructuring  the  program  must  lay  down  a  con- 
vincing set  of  arguments  that  change  will  be  beneficial. 

This  is  particularly  true  since  there  seems  to  be  agreement  that 
a  principal  role  of  the  program  remains  unchanged.  That  is,  to  pro- 
vide access  to  high-end  computing  infrastructure  for  the  academic 
science  and  engineering  community. 

I  look  forward  to  review  of  this  Supercomputer  Centers  Program, 
and  in  particular  to  a  discussion  of  how  the  program  may  be 
strengthened  and  how  the  proposed  changes  will  affect  the  re- 
search community,  which  has  come  to  rely  on  this  important  source 
for  the  conduct  of  science  and  engineering  research. 

I  look  forward  to  the  subcommittee  hearing  today. 

Mr.  SCHIFF.  Thank  you,  Mr.  Cramer. 

I  want,  before  recognizing  other  Members,  I  just  want  to  say 
briefly  that  I  want  to  affirm  to  you  publicly  what  I  have  said  pri- 
vately. 


During  the  six  years  I  served  on  the  Science  Committee  under 
the  Democratic  Majority,  I  always  felt  treated  very  fairly  by  the 
committee  chairman,  and  by  the  subcommittee  chairs,  and  I  always 
felt  that  partisanship  never  entered  into  this  Committee,  and  that 
if  we  had  disagreements  they  were  on  the  merits  as  we  saw  mat- 
ters, and  I  want  to  assure  you  of  my  best  intention  to  continue  that 
precedent. 

With  that 

Mr.  Cramer.  Thank  you. 

Mr.  SCHIFF.  I  note  that  we  are  joined  by  our  Ranking  Democratic 
Member,  Congressman  Brown  from  California,  who  I  recognize  for 
as  much  time  as  he  wishes  to  consume. 

[Bells  ring.] 

Mr.  Brown.  Thank  you,  Mr.  Chairman.  I  will  not  consume  a 
great  deal  of  time 

[Bells  begin  to  ring  again.] 

[Laughter.] 

Mr.  Brown.  Uh-oh. 

Mr.  SCHIFF.  I  think  that  is  a  recess. 

Mr.  Brown.  Two  weeks  ago  there  was  a  weekend  program  to 
wire  up  as  many  schools  as  possible  in  California  to  the  Internet, 
which  I  thought  was  a  marvelous  PR  thing,  and  actually  did  hook 
up  quite  a  few  schools. 

Yesterday  morning  I  visited  one  of  those  schools  and  talked  to  an 
8th  grade  class,  or  rather  watched  them  operating  their  computers 
and  accessing  the  Worldwide  Web  and  doing  a  lot  of  other  things 
that  I  thought  were  incomprehensible,  or  probably  would  not  ever 
have  occurred. 

This  morning  I  met  with  the  representatives  of  most  of  the  major 
cable  companies  in  California. 

Now  I  cite  these  three  things  to  illustrate  the  changes  that  have 
taken  place  in  the  relatively  recent  past. 

Many  of  these  changes  I  think  can  be  attributed  to  this  program 
which  has  developed  the  capability  for  making  better  use  of  com- 
puters in  our  institutions  of  higher  learning,  as  well  as  our  institu- 
tions of  lower  learning. 

It  is  one  of  the  marvels  of  my  experience  here  in  Congress  to 
have  been  a  part  of  this  development  over  these  last  10  or  15  years. 

The  next  10  or  15  years  can  be  equally  exciting.  I  won't  be  here 
for  most  of  that  time,  but  many  of  you  will  and  I  urge  you  to  take 
the  same  care  and  concern  over  the  further  development  of  these 
capabilities  that  have  proven  to  be  such  a  successful  activity,  and 
one  in  which  the  committee  has  had  a  fairly  important  role  over 
the  last  10  or  15  years. 

It  can  give  you  great  satisfaction  in  your  work,  I  can  assure  you. 

Thank  you,  Mr.  Chairman. 

Mr.  SCHIFF.  Thank  you,  Mr.  Brown. 

Does  any  other  Member  seek  recognition  for  any  opening  state- 
ment? 

Mr.  Ehlers? 

Mr.  Ehlers.  Thank  you,  Mr.  Chairman.  Just  briefly,  several 
items. 

First  of  all,  I  am  very  impressed  with  the  panel  you  have  assem- 
bled and  I  look  forward  to  the  hearing.  I  apologize  that  I  have  to 


go  to  the  Floor  at  some  point  to  defend  a  bill  that  is  up  for  consid- 
eration. 

The  Chairman  of  the  House  Oversight  Committee  is  tied  up  in 
another  committee,  so  I  have  to  go  there  to  defend  it,  and  I  apolo- 
gize to  the  panel  when  I  am  absent. 

The  other  point  I  would  make  is.  A  few  weeks  ago  I  circulated 
a  letter  among  my  colleagues  in  the  matter  and  in  just  a  few  days 
received  89  signatures  asking  that  we  immediately  put  in  place 
full-year  funding  for  the  National  Science  Foundation. 

I  was  assured  that  this  would  receive  serious  consideration,  and 
in  fact  it  did,  but  it  has  been  held  up  because  there  is  an  effort  to 
provide  full-year  funding  for  every  part  of  the  government  that  is 
not  fully  funded  at  this  point. 

So  that  issue  is  in  abeyance. 

I  simply  want  to  get  this  on  the  record.  Once  again,  if  for  some 
reason  we  are  not  able  to  complete  work  on  the  Fiscal  1986  budget 
soon,  I  believe  it  imperative  that  we  provide  full-year  funding  for 
the  National  Science  Foundation  as  soon  as  possible,  since  they  op- 
erate largely  through  grants,  and  many  grants  are  being  held  up 
pending  completion  of  the  budget,  and  many  of  the  individuals  in 
this  room,  plus  thousands  of  others  throughout  the  country,  are 
anxiously  awaiting  confirmation  of  their  grguits  and  final  award  of 
the  grants. 

I  think  it  is  extremely  important  for  us  to  recognize  as  a  Con- 
gress and  as  a  Nation  the  important  role  that  science  plays  in  the 
infrastructure  for  our  economic  engine.  What  we  are  about  to  hear 
today  says  the  same  thing. 

The  Ranking  Member  of  the  Full  Committee,  Mr.  Brown,  for 
years  has  been  a  strong  advocate  not  only  of  science  but  particu- 
larly of  the  supercomputing  centers,  as  well  as  coordination  of  com- 
puting activities  among  various  agencies. 

We  are  trying  to  carry  on  that  mandate,  and  it  has  become  ex- 
tremely difficult  because  of  the  uncertainty  of  funding. 

Now,  Mr.  Chairman,  I  believe  it  is  very  important  that  we  get 
the  funding  issue  settled  as  soon  as  possible  and  ensure  that  we 
continue  as  a  Nation  to  have  the  computing  capability  we  need  to 
carry  on  the  scientific  enterprise  in  this  Nation. 

I  thank  you  for  holding  the  hearing,  and  thank  you  especially  for 
assembling  the  panel  that  you  have. 

Thank  you. 

Mr.  SCHIFF.  Thank  you,  Mr.  Ehlers. 

As  one  of  many  of  the  dozens  of  lawyers  in  Congress,  I  am 
pleased  to  have  a  real  scientist  here  on  the  subcommittee. 

Before  I  recognize  more  Members,  I  want  to  add  two  things  very 
briefly.  The  first  is.  I  join  you  in  the  belief  that  if  we  are  not  able 
to  fund  all  agencies  for  the  balance  of  the  fiscal  year,  that  we 
should  identify  and  fund  the  National  Science  Foundation  sepa- 
rately and  would  support  that,  and  I  appreciate  your  leadership  in 
it. 

The  second  is,  as  you  mentioned  you  have  to  go  to  the  House 
Floor  in  a  bit  to  speak  on  behalf  of  a  bill,  so  do  I.  I  want  to  express 
to  those  of  you  who  are  here  who  may  never  have  seen  a  Congres- 
sional hearing  before,  those  of  you  who  have  been  here  before  are 


quite  accustomed  to  the  fact  that  we  schedule  several  things  at  any 
one  time. 

Those  of  you  who  have  never  been  to  a  hearing  before  I  under- 
stand can  be  a  little  disconcerted  at  our  coming  and  going,  but 
what  I  want  to  emphasize  is  that  the  main  purpose  of  a  hearing 
is  to  record  the  information  so  in  due  course  it  is  made  available 
to  all  Members  of  Congress. 

And  of  course  the  most  important  person  at  the  hearing,  there- 
fore, is  the  lady  who  is  taking  down  everything  that  we  are  saying 
back  and  forth  because,  regardless  of  who  is  sitting  in  this  chair 
or  the  other  chairs,  all  that  information  gets  across  to  the  entire 
Congress. 

So  I  want  to  assure  you  of  the  importance  we  place  on  this  hear- 
ing, even  if  at  times  we  may  have  to  personally  leave  it  for  a  bit. 

Let  me  now  recognize  the  gentleman  from  Pennsylvania,  Con- 
gressman Doyle. 

Mr.  Doyle.  Thank  you,  very  much. 

Mr.  Chairman,  I  want  to  thank  you  for  holding  today's  hearing 
on  the  National  Science  Foundation's  Supercomputing  Centers  Pro- 
gram. 

As  a  Member  who  is  fortunate  enough  to  have  one  of  these  cen- 
ters in  my  District,  I  am  well  aware  of  the  contribution  that  they 
have  made  and  will  continue  to  make  to  our  Nation's  technological 
infrastructure. 

I  do  want  to  start  by  welcoming  our  witness  from  the  Pittsburgh 
Center,  Dr.  Ralph  Roskies.  He,  along  with  many  other  dedicated  in- 
dividuals, has  made  the  Pittsburgh  Supercomputing  Center  an 
internationally  recognized  success. 

The  PSC  has  made  meaningful  contributions  in  many  areas-no- 
tably  in  assisting  NIH  research,  developing  weather  models  for  the 
National  Weather  Service,  and  helping  EPA  with  its  Air  Quality 
Assessments. 

Today's  hearing  examines  the  status  of  all  the  Supercomputing 
Centers,  which  I  believe  is  a  timely  and  worthwhile  undertaking. 

We  have  before  us  three  relatively  new  reports,  the  most  recent 
of  which,  the  Hayes  Report,  calls  for  some  rather  significant  action 
to  be  taken  on  this  program. 

I  am  hopeful  that  today's  hearing  will  give  us  an  opportunity  to 
understand  why  the  Hayes  Report  calls  for  a  departure  from  what 
has  been  an  unquestionably  successful  program. 

Again,  I  am  pleased  that  we  are  having  this  hearing.  I  hope  it 
will  allow  us  to  understand  the  significance  of  this  program  prior 
to  acting  in  any  way  to  dismantle  it. 

Thank  you  very  much,  Mr.  Chairman. 

Mr.  SCHIFF.  Thank  you,  Mr.  Doyle.  I  wanted  to  appreciate  your 
efforts  on  behalf  of  supercomputing  both  on  behalf  of  the  Pitts- 
burgh Center  and  on  behalf  of  the  issue  generally.  It  has  been  a 
good  contribution  to  this  subcommittee. 

Mr.  Doyle.  Thank  you,  Mr.  Chairman. 

Mr.  SCHIFF.  Now  I  want  to  say  that  all  Members'  statements, 
without  objection,  will  be  made  part  of  the  record  but  if  any  other 
Member  desires  to  make  an  opening  oral  statement  I  will  recognize 
them  at  this  time. 

[No  response.] 


Mr.  SCHIFF.  I  see  no  requests,  so  I  am  going  to  invite  our  first 
panel. 

Dr.  Young  and  Dr.  Hayes,  please  joint  us  at  the  witness  table. 

Gentlemen,  while  you  are  getting  set  up  there,  let  me  say  that 
your  complete  written  statements  will  be  made  a  permanent  part 
of  this  record  and  invite  you  to  summarize  or  proceed  as  you  think 
best. 

Dr.  Young,  please  proceed. 

STATEMENT  OF  DR.  PAUL  YOUNG,  ASSISTANT  DIRECTOR  FOR 
COMPUTER  AND  INFORMATION  SCIENCE  AND  ENGINEER- 
ING, NATIONAL  SCIENCE  FOUNDATION 

Dr.  Young.  All  right.  I  appreciate  that,  and  I 

Mr.  SCHIFF.  I  think  your  microphone  may  not  be  on. 

Dr.  Young.  Thank  you.  I  appreciate  that,  and  I  will  limit  myself 
to  oral  comments  that  I've  made. 

Chairman  Schiff,  Members,  staff,  guests: 

I  am  Paul  Young,  Assistant  Director  of  the  National  Science 
Foundation's  Computer  and  Information  Science  and  Engineering. 

I  would  like  to  begin  by  commenting  how  much  I,  and  I  am  sure 
all  members  of  the  Foundation  and  members  of  the  science  and  en- 
gineering communities  generally,  appreciate  the  comments  of  sup- 
port and  the  efforts  by  Mr.  Ehlers,  by  chairman  Schiff,  and  by  this 
committee  as  a  whole  in  support  of  both  the  1996  budget,  and  basic 
research  and  science  and  engineering  generally. 

My  Directorate  at  the  Foundation  supports  both  research  by,  and 
infrastructure,  for  the  scientific  and  engineering  communities.  I  am 
here  today  to  discuss  our  new  program,  Partnerships  in  Advanced 
Computational  Infrastructure,  which  is  a  follow-on  to  the  very  suc- 
cessful NSF  Supercomputer  Centers  Program  begun  in  1984. 

Our  research  programs  in  computer  science  and  engineering  sup- 
port basic  research  in  computing  information  and  communications. 
Our  organization's  other  infi-astructure  program  is  in  networking, 
a  program  which  has  led  to  the  current  development  of  the 
Internet,  a  subject  about  which  you  see  daily  references  in  the 
press  and  elsewhere,  and  indeed  Mr.  Brown  referenced  the  current 
linking  of  the  California  schools,  and  we  are  very  enthused  about 
that  as  well. 

It  is  interesting  to  note  that  our  two  infi-astructure  programs 
began  in  the  same  NSF  office  about  1984  and  have  continued  to 
support  and  complement  each  other  to  this  day. 

I  am  going  to  first  talk  a  bit  about  the  background  that  led  to 
the  establishment  of  the  Supercomputer  Centers  Program,  and 
then  briefly  review  some  of  its  successes  and  the  reasons  for  them. 

I  will  then  discuss  the  process  that  led  to  the  restructured  pro- 
gram and  its  goals. 

And  finally,  I  would  like  to  lay  out  the  process  and  time  line  for 
putting  our  new  program  in  place  and  our  commitment  to  a  smooth 
transition,  a  transition  which  will  minimize  the  impact  on  the  com- 
munity of  scientists  and  engineers  who  depend  on  us  to  support 
their  research. 

When  I  finish,  I  would  like  to  leave  you  with  the  following  points: 

First,  the  Supercomputer  Centers  Program  was  created  in  re- 
sponse to  a  critical  need  of  the  research  and  education  commu- 
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nities.  It  has  not  only  succeeded  in  meeting  that  need,  it  has  gone 
well  beyond. 

Second,  the  science  and  engineering  enabled  by  the  program  is 
of  extremely  high  quality.  The  need  for  continued  support  remains, 
and  is  in  fact  increasing  as  computation  becomes  increasingly  im- 
portant in  more  and  more  research  areas. 

Third,  advances  in  technology  will  now  enable  us  to  restructure 
the  program  to  involve  a  broader  spectrum  of  individuals  and  insti- 
tutions, chosen  in  a  competition  where  the  best  ideas  will  be  sought 
and  implemented. 

Fourth,  we  at  NSF  are  committed  to  a  fair  and  open  competition 
using  the  best  expertise  in  industry,  academia,  and  government  to 
review  proposals. 

And  finally,  every  effort  will  be  made  to  maintain  quality  service 
to  the  research  community  during  the  transition  to  the  new  pro- 
gram. 

The  history  of  the  Supercomputer  Centers  Program  can  be  traced 
back  to  the  efforts  of  a  number  of  people  who  realized  that  com- 
putation could  provide  an  important  tool  for  research  in  science 
and  engineering,  who  also  realized  that  advanced  computation  was 
not  generally  available  to  the  research  communities  that  NSF  tra- 
ditionally supports. 

In  the  early  1980s,  NASA  and  DOE  scientists  had  access  to  ad- 
vanced computational  tools  for  programmatic  needs.  NSF  research- 
ers whose  work  was  related  to  these  programmatic  agency  goals 
could,  in  some  cases  get,  access,  but  others  could  not. 

For  example,  there  were  cases  at  that  time  where  the  only  way 
that  NSF  researchers  could  do  their  computational  work  was  to  go 
overseas. 

The  realization  of  the  need  for  computation  to  support  a  broad 
range  of  research  in  science  and  engineering  and  the  support  of 
many  in  Congress  and  in  the  Administration  led  to  the  establish- 
ment of,  first  an  access  program;  and  then  the  NSF  Supercomputer 
Centers  in  the  1984-85  time  range. 

To  say  that  this  program  has  been  a  success  is  truly  an  under- 
statement. Things  that  were  not  even  imagined  at  the  beginning 
have  been  accomplished  in  the  fields  of  science  and  engineering 
that  no  one  envisioned  as  having  any  need  for  computation  have 
been  positively  impacted. 

It  is  often  said  that  success  has  many  parents  and  failure  is  an 
orphan.  If  this  can  be  quantified,  then  the  degree  of  success  must 
be  related  to  the  number  of  people  who  have  been  a  part  of  the  en- 
terprise over  the  last  11  years. 

There  are  a  multitude.  Some  of  them  are  here.  But  since  the  in- 
ception of  the  program,  more  than  20,000  people  have  used  the  pro- 
gram in  support  of  their  research,  and  over  100  industrial  firms 
have  used  the  program  for  training  and  to  test  the  applicability  of 
supercomputing  for  their  firms. 

In  one  recent  year  alone,  over  20,000  people  had  contact  with  the 
Centers'  education  and  training  programs.  Let  me  give  a  few  exam- 
ples of  technology  and  research  activities  associated  with  the  pro- 
gram, but  remind  you  that  later  speakers  will  continue  to  provide 
instances  of  major  successes  in  research  and  education. 


One  important  far-reaching  development  in  the  success  of  the 
Centers  was  the  reahzation  that  researchers  needed  access  from 
their  home  institutions.  Originally,  several  of  the  Centers  even  ran 
their  own  networks;  but  as  interest  broadened,  it  became  clear  that 
a  national  network  was  required. 

The  Centers  were  the  original  nodes  on  the  NSFNet's  National 
Network,  and  the  original  nodes  on  the  high-speed  Net  backbone, 
and  as  such  they  played  a  major  role  in  driving  the  networking 
technologies  needed  for  today's  Internet. 

Today  they  are  playing  a  similar  role  in  the  evolution  of  NSF's 
experimental,  very  high-speed  network,  the  VBNS. 

Another  tool  that  assumed  importance  came  from  the  realization 
that  the  results  of  complex  calculations  could  not  be  understood 
from  conventional  computer  output.  The  program  has  dramatically 
enhanced  the  development  of  software  tools  to  visualize  the  results 
of  calculations. 

With  the  network  connection,  access  to  high-speed  computers, 
and  visualization  tools,  computational  scientists  and  engineers 
were  empowered  to  follow  their  imagination  to  new  results  and  to 
new  understandings. 

What  are  some  of  those  research  areas?  And  how  is  the  field  de- 
veloping? 

As  one  example,  Gerry  Ostriker,  a  cosmologist,  who  is  now  the 
Provost  at  Princeton  University,  in  briefing  the  National  Science 
Board  and  the  Hayes  Task  Force,  said  that  years  ago  he  believed 
that  computers  were  having  a  negative  impact  on  his  area  of  re- 
search. Promising  graduate  students  would  get  intrigued  with  at- 
tacking fi-ont-line  physics  problems  with  underpowered  computers 
and  software,  and  then  get  absorbed  in  the  technology. 

Since  then,  the  power  of  the  machines  and  the  understanding  of 
programming  them  have  made  enormous  strides.  Today,  Ostriker 
is  one  of  the  principal  investigators  in  a  team  that  has  made  im- 
portant progress  in  modeling  the  formation  of  the  universe. 

The  models  are  now  so  good  that  they  can  differentiate  between 
competing  theories  of  the  universe,  and  focus  both  theoretical  and 
observational  work  on  the  most  fruitful  lines  of  inquiry. 

Another  rapidly  growing  area  of  computation  is  in  biology,  where 
physiologically  interesting  molecules  can  be  successfully  modelled 
by  computational  methods  and  their  behavior  understood. 

It  is  more  and  more  likely  that  biologically  active  molecules — for 
example,  new  drugs — can  be  designed  and  their  behavior,  both 
good  and  bad,  predicted  with  computational  models. 

Climate  and  meteorology  research  continue  to  benefit  from  com- 
puting. Later  this  afternoon  you  will  hear  about  storm  prediction. 

Work  is  also  going  on  trying  to  understand  the  effects  of  green- 
house gases  on  long-term  climate  change.  These  calculations  re- 
quire a  sophisticated  understanding  of  the  interaction  of  the  atmos- 
phere and  the  ocean,  which  will  have  important  other  major  bene- 
fits to  all  of  us. 

Every  time  we  think  we  understand  which  fields  can  benefit  from 
computational  results,  a  new  threshold  in  computing  capability  is 
crossed  and  another  field  blossoms.  As  our  progress  has  continued, 
it  can  be  truly  said  that  computation  has  become  a  full  partner  in 
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the  scientific  method,  along  with  experimentation,  theory,  and  ob- 
servation. 

In  1993  a  blue-ribbon  panel  chaired  by  Lou  Branscomb  issued  a 
report  on  The  Future  of  Computation  and  the  Tools  Needed  To  Ex- 
ploit It.  Along  with  a  truly  ringing  endorsement  of  the  accomplish- 
ments of  the  NSF's  Centers  Program,  this  report  called  for  distrib- 
uting the  technology  across  a  pyramid  of  capability  from  the  apex 
to  work  stations. 

At  the  time  of  the  Branscomb  report,  the  budget  of  the  national 
High  Performance  Computing  and  Communications  Program  was 
predicted  to  double  in  five  years.  While  the  Branscomb  Report  was 
well  received,  budget  realities  prevailed. 

The  National  Science  Board,  when  confi-onted  with  a  proposal  to 
continue  the  Centers  without  recompetition,  was  reluctant.  Later 
that  year,  the  Board  approved  a  two-year  extension  to  the  program 
while  a  task  force  chaired  by  Ed  Hayes  evaluated  a  direction  that 
was  more  consistent  both  with  budget  predictions  and  with  a  rap- 
idly evolving  technology. 

The  Task  Force  on  The  Future  of  the  NSF  Supercomputers  Pro- 
gram met  and  conferred  during  calendar  1995,  issuing  a  final  re- 
port to  the  NSF  Director  in  September.  The  Task  Force  was  an 
independent  group  with  many  sectors  represented,  and  it  dealt 
with  the  really  hard  issues. 

The  next  speaker,  Ed  Hayes,  was  chairman  of  that  Task  Force 
and  will  speak  to  its  conclusions  and  the  report.  Based  on  the 
Hayes  Report  and  on  deliberations  within  the  Foundation,  NSF 
management  decided  in  late  1995  to  put  forth  a  new  program  to 
the  National  Science  Board  which  was  approved  at  their  December 
1995  meeting. 

It  is  designed  to  officially  take  advantage  of  new  partnerships, 
explicitly  including  regional  and  state  partnerships,  and  these  in 
turn  take  advantage  of  and  drive  both  scalable  parallel  computing 
and  new  advanced  networking  capabilities.  The  program  solicita- 
tion for  the  new  program  was  released  on  the  Worldwide  Web  just 
prior — literally,  the  day  of — the  government  shutdown  on  Decem- 
ber 15th,  1995. 

Later,  a  formal  printed  solicitation  and  a  "Dear  Colleague"  letter 
were  issued,  and  a  series  of  informational  meetings  were  held  on 
both  Coasts  and  in  the  Midwest. 

As  evidence  of  the  accessibility  of  this  information,  and  in  spite 
of  the  shutdown,  our  records  show  that  over  1000  accesses  were 
made  to  the  Web  information  before  the  shutdown  ended,  and  that 
over  3000  accesses  have  been  made  to  the  Hayes  report  and  the  so- 
licitation from  over  2000  distinct  Internet  addresses. 

The  current  Centers  Program  and  the  new  Partnerships  Program 
are  designed  to  support  the  academic  research  community.  As  a 
general  rule,  proposals  and  principal  investigators  at  NSF  are 
drawn  from  academia.  This  is  pointed  out  in  the  solicitation. 

However,  NSF  is  open  to  good  ideas  from  any  sector,  and  the  cur- 
rent program  involves  industry  successfully  in  a  variety  of  ways. 
We  will  be  pleased  to  see  such  involvement  continue  in  the  future, 
and  we  will  carefully  review  any  new  proposal  that  meets  NSF's 
broad  acceptance  criteria. 
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The  time  line  and  the  process  for  a  fair  evaluation  of  proposals 
is  painstakingly  detailed  in  our  written  testimony,  and  the  solicita- 
tion and  the  executive  summary  of  the  Hayes  report  are  also  in- 
cluded. 

In  my  opinion,  the  NSF  has  listened  carefully  to  its  constituents, 
has  paid  close  attention  to  the  scientific  needs  of  the  Nation,  and 
is  going  forward  with  the  program  designed  to  capitalize  on  tech- 
nical and  budget  realities. 

We  feel  very  confident  that  the  new  proposals  we  receive  will  be 
very  innovative,  and  they  will  be  carefully  reviewed  using  NSF's 
best  traditions  of  peer  review. 

We  expect  to  go  forward  with  a  program  that  the  Nation  can  be 
proud  of,  and  one  that  will  continue  the  unparalleled  successes  of 
the  past. 

Thank  you,  Mr.  Chairman. 

[The  prepared  statement  of  Dr.  Young  follows:] 
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Mr.  Chairman  and  Members  of  the  Subcommittee,  thank  you  for  the 
opportunity  to  testify  here  today  on  NSF's  new  program  of  Partnerships  for 
Advanced  Computational  Infrastructure  (PACI). 

I  am  Paul  Young,  Assistant  Director  for  NSF's  Directorate  for  Computer  and 
Information  Science  and  Engineering  (CISE). 

The  new  PACI  program,  which  replaces  the  successful  NSF  Supercomputer 
Centers  program,  is  one  important  element  in  the  Foundation's 
implementation  of  the  National  High  Performance  Computing  and 
Communications  (HPCC)  Program. 

At  NSF  the  HPCC  Program  focuses  on  advancing  the  full  range  of  advanced 
computing,  communications,  information  technologies,  and  infrastructure, 
and  supports  fundamental  research  in  these  areas  of  computer  science  and 
engineering,  as  well  as  the  great  diversity  of  applications  in  all  areas  of  science 
and  engineering  research  which  require  HPCC.   Thus,  the  research  in 
computer  science  and  engineering  supported  in  CISE  is  the  prerequisite  to  the 
development  of  future  high  performance  computing,  communications  and 
information  systems,  and  forms  the  basis  of  an  advanced  National 
Information  Infrastructure.    CISE  also  provides  computing  and 
communications  infrastructure  for  all  research  and  education  supported  by 
the  Foundation  through  support  for  the  Internet  and  the  Supercomputer 
Centers. 

In  this  context,  we  expect  that  the  new  PACI  Program  will  benefit  from,  and 
provide  new  opportunities  in,  future  high  performance  computing  and 
communications  activities  throughout  the  nation. 

NSF's  Early  Participation  in  High  Performance  Computing 

The  National  Science  Foundation  is  now  a  major  partner  in  providing  the 
nation's  high  performance  computing  infrastructure,  but  this  was  not  always 
the  case.  In  the  early  1970s  the  NSF  ceased  its  support  of  campus  computing 
centers,  and  by  the  mid-1970s  there  were  no  "supercomputers"  generally 
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a\-ailable  to  the  academic  community  on  any  campus.  Although  computers  of 
this  capability  were  available  through  other  government  agency  (DoE  and 
NASA)  laboratories,  NSF  did  not  play  a  role.  As  a  consequence,  most 
academic  researchers  did  not  have  the  ability  to  perform  computational 
research  on  anything  other  than  a  departmental  minicomputer,  thereby 
limiting  the  scope  of  research  in  many  fields  of  science  and  engineering. 

This  lack  of  access  to  high  performance  computing  was  noted  in  the  early 
1980's  in  of  a  growing  number  of  reports.  For  example  a  report  to  the  NSF 
Division  of  Phvsics  Advisory  Committee  in  March  1981  entitled  "Prospectus 
for  Computational  Physics",  edited  by  William  Press,  identified  a  "crisis"  in 
computational  physics,  and  recommended  support  for  facilities.  Subsequent 
to  this  report  a  joint  agency  study,  "Large  Scale  Computing  in  Science  and 
Engineering",  edited  by  Peter  Lax,  appeared  in  December  1982  and  acted  as  the 
catalyst  for  NSF's  reemergence  in  the  support  of  high  performance 
computing.  The  Lax  Report  presented  four  recommendations  for  a 
government-wide  program: 

•  Increased  access  to  regularly  upgraded  supercomputing  facilities  via  high 
bandwidth  networks 

•  Increased  research  in  computational  mathematics,  software,  and 
algorithms 

•  Training  of  personnel  in  scientific  computing 

•  R&D  for  new  supercomputer  systems 

The  key  suggestions  contained  in  the  Lax  Report  were  studied  by  an  internal 
NSF  working  group,  and  the  findings  were  issued  in  July  1983  as  "A  National 
Computing  Environment  for  Academic  Research",  a  report  edited  by  M. 
Bardon  and  K.  Curtis.  The  report  studied  NSF  supported  scientists'  needs  for 
academic  computing,  and  validated  the  conclusions  of  the  Lax  Report  for  the 
NSF  supported  research  community.  The  findings  of  Bardon/Curtis 
reformulated  the  four  recommendations  of  the  Lax  Report  into  a  six  point 
implementation  plan  for  the  NSF.  Part  of  this  action  plan  was  a 
recommendation  to  establish  ten  academic  supercomputer  centers. 

The  immediate  NSF  response  was  to  set  up  a  means  for  academic  researchers 
to  have  access,  at  existing  sites,  to  the  most  powerful  computers  of  the  day. 
This  was  an  interim  step  prior  to  a  solicitation  for  the  formation  of  academic 
supercomputer  centers  directly  supported  by  the  NSF.  By  1987,  five  NSF 
Supercomputer  Centers  had  been  established,  and  all  had  completed  at  least 
one  year  of  operation. 

During  this  early  phase  the  Centers  were  essentially  isolated  "islands  of 
supercomputing"  whose  role  was  to  provide  supercomputer  access  to  the 
academic  community.  This  aspect  of  the  Centers'  activities  has  changed 
considerably.  The  NSF  concept  of  the  Centers'  activities  was  mandated  to  be 
much  broader,  as  indicated  by  the  Center's  original  objectives  including: 
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•  Access  to  state  of  the  art  supercomputers 

•  Nurture  computational  science  and  engmeermg  m  all  fields 

•  Traming  of  computational  scientists  and  engineers 

•  Encourage  collaboration  among  researchers  in  academia,  industry  and 
government 

In  1988-1989  NSF  conducted  a  review  to  determine  whether  support  was 
justified  bevond  1990.  In  developing  proposals,  the  NSF  Centers  were  advised 
to  increase  their  scope  of  responsibilities.  Quoting  from  the  solicitation: 

"To  insure  the  long  term  health  and  value  of  a  supercomputer  center,  an 
intellectual  environment,  as  well  as  first  class  service,  is  necessary.  Centers 
should  identify  an  intellectual  component  and  research  agenda". 

In  1989  NSF  approved  continuation  through  1995  of  the  Cornell  Theory 
Center,  the  National  Center  for  Supercomputing  Applications,  the  Pittsburgh 
Supercomputing  Center,  and  the  San  Diego  Supercomputer  Center.  Support 
for  the  John  von  Neumann  Center  located  at  Princeton  University  vvas  not 
continued. 

Scientific  accomplishments  of  the  Program 

The  Centers  have  fostered  fundamental  advances  in  our  understanding  of 
science  and  engineering,  expanded  the  use  of  high-end  computing  in  new 
disciplines,  enabled  the  major  paradigm  shift  to  the  acceptance  of 
computational  science  as  a  full  partner  in  the  scientific  method,  and 
facilitated  the  education  of  a  new  generation  of  computational  scientists  and 
engineers  in  support  of  that  shift. 

Several  common  themes  emerge  from  examples  of  the  impact  of 
supercomputing. 

•  First,  the  rapid  growth  of  supercomputing  together  with  its  availability  to 
the  research  community  have  enabled  computational  science  and 
engineering  to  contribute  to  significant  advances  in  a  very  wide  and  still 
growing  set  of  scientific  and  engineering  fields. 

•  Second,  high  performance  computing  is  making  it  possible  to  perform 
complex  simulations  in  three  dimensions,  rather  than  just  two.    This 
important  shift  has  dramatically  enhanced  the  usefulness  of 
computational  approaches,  and  will  continue  to  do  so  at  least  through  the 
coming  decade. 

•  Third,  supercomputer-based  simulations  have  combined  multiple 
disciplines  and  different  physical  phenomena  to  yield  new  scientific 
discoveries  and  understanding. 

•  Last,  increases  in  supercomputing  capability  and  advances  in 
computational  techniques  are  beginning  to  enable  computer-based 
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bimulations  to  predict  new  scientific  advances  and  to  make  new 
discoveries. 

In  later  testimony  at  this  hearing  the  current  Center  Directors  will  highlight 
the  scientific  and  engineering  accomplishments  of  the  program  in  the  last  11 
vears. 

Relevant  Reports 

Report  of  the  Blue  Ribbon  Panel 

Following  the  renewal  of  four  of  the  Centers  in  1990,  the  National  Science 
Board  (NSB)  asked  the  director  of  NSF  to  appoint  a  blue  ribbon  panel  "...   to 
investigate  the  future  changes  in  the  overall  scientific  environment  due  [to] 
the  rapid  advances  occurring  in  the  field  of  computers  and  scientific 
computing."  The  resulting  report,  "From  Desktop  to  Teraflop:  Exploiting  the 
U.S.   Lead  in  High  Performance  Computing,"  was  edited  by  Lewis  Branscomb, 
and  was  presented  to  the  NSB  in  October,  1993. 

This  report  pointed  to  the  Foundation's  accomplishments  in  the  seven  years 
since  the  initial  implementation  of  the  recommendations  of  the  Lav  Rr^port 
on  high  performance  computing  (HPC)  and  the  establishment  of  the 
Supercomputer  Centers.   The  report  asserted  that  the  NSF  Centers  had  created 
an  enthusiastic  and  demanding  set  of  sophisticated  users  who  make 
fundamental  advancements  in  their  scientific  and  engineering  disciplines 
through  the  application  of  rapidly  evolving  high  performance  computing 
technology.   Other  measures  of  success  cited  include  the  thousands  of 
researchers  and  engineers  who  have  gained  experience  in  HPC,  and  the 
extraordinary  technical  progress  in  realizing  new  computing  environments. 

The  report  noted  that,  through  the  NSF  program  and  those  of  sister  agencies, 
the  U.  S.    enjoys  a  substantial  lead  in  computational  science  and  in  the 
emergmg,  enabling  technologies.   It  called  for  NSF  to  capitalize  on  this  lead, 
which  not  only  offers  scientific  preeminence,  but  also  aids  the  associated 
industrial  lead  in  many  growing  world  markets. 

Primary  recommendations  included  the  following: 

•  The  NSF  should  retain  the  Centers  and  reaffirm  their  mission  with  an 
understanding  that  they  now  participate  in  a  much  richer 
computational  infrastructure  than  existed  at  their  formation. 

•  The  NSF  should  assist  the  university  community  in  acquiring  mid- 
range  systems  to  support  scientific  and  engineering  computation  and  to 
break  down  the  software  barriers  associated  with  massively  parallel 
systems. 

•  The  NSF  should  initiate  an  interagency  plan  to  provide  a  balanced 
teraflop  system,  with  appropriate  software  and  computational  tools,  at 
the  apex  of  the  computational  pyramid. 
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These  recommendations  and  the  accompanying  challenges  could  be 
summarized  as  calling  for  a  broad  based  infrastructure  and  research  program 
that  would  not  only  support  the  range  of  computational  needs  required  by  the 
existing  user  base,  but  would  also  broaden  that  base  in  terms  of  the  range  of 
capabilities,  expertise,  and  disciplines  supported. 

Report  of  the  NRC-HPCC  Committee 

In  1994,  Congress  asked  the  National  Research  Council  (NRC)  to  examine  the 
status  of  the  High  Performance  Computing  and  Communications  Initiative. 
The  NRC  committee  stated  that  it  "believes  that  strong  public  support  for  a 
broadly  based  research  program  in  information  technology  is  vital  to 
maintaining  U.S.  leadership  in  information  technology".  While  the 
committee  did  not  make  explicit  recommendations  for  funding  levels  or 
management  structures  for  the  Supercomputer  Centers  program,   it  did  say: 

"The  committee  recognizes  that  advanced  computation  is  an 
important  tool  for  scientists  and  engineers  and  that  support  for 
adequate  computer  access  must  be  a  part  of  the  NSF  research  program 
in  all  disciplines.    The  committee  also  sees  value  in  providing  large- 
scale,  centralized  computing,  storage,  and  visualization  resources  that 
can  provide  unique  capabilities.   How  such  resources  should  be  funded 
and  what  the  long  term  role  of  the  Centers  should  be  with  respect  to 
both  new  and  maturing  computing  architectures  are  critical  questions 
that  NSF  should  reexamine  in  detail,  perhaps  via  the  newly 
announced  Ad  Hoc  Task  Force  on  the  Future  of  the  NSF 
Supercomputer  Centers  Program." 

The  Task  Force  on  the  Future  of  the  NSF  Supercomputer  Centers  Program 

The  Task  Force  met  during  NSF's  1995  Fiscal  Year  to  advise  NSF  on  several 
important  issues  related  to  the  review  and  management  of  the  NSF 
Supercomputer  Centers  program.   The  Task  Force  was  charged  to  analyze 
various  alternatives  for  the  continuation,  restructuring,  or  phase-out  of 
NSF's  current  Supercomputer  Centers  program,  or  the  development  of 
similar  future  program(s),  and  to  make  recommendations  among  the 
alternatives. 

The  final  report  of  the  Task  Force  was  presented  to  the  NSF  Director  on 
September  15,  1995.  The  following  language  is  excerpted  from  the  executive 
summary  of  the  report,  which  is  appended  to  this  testimony. 

The  Task  Force  believes  that  the  future  for  computational  science  and 
engineering  can  be  as  bright  or  even  brighter  than  in  the  past  decade.    If  we 
seize  the  opportunity,  over  the  next  decade  we  can  make  major  progress  on 

multiple    fronts. 

There   will  be 

•    opportunities  for  exciting  applications  of  our  nation's  exponentially 
increasing  computational  capacity,  for  example: 
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-  more  complete   models,   and  hence  deeper   understanding  of 
physical  systems   by  moving  to   three  and  higher  dimensions; 

-  progress  in  computational  tools  to  aid  drug  and  protein  design; 

-  computational   predictions   of  scientifically   and   commercially 
significant    materials; 

-  multidisciplinary   models  of  physical  systems   (e.g.,   combining 
fluid  dynamics  and  electromagnetic  models  of  the  heart); 

-  increased   interconnectivity  of  supercomputers   and   high    impact 
instrumentation;    and 

-  models  of  anatomical  and  physiological  processes  leading  to  nezv 
insights  of  benefit   to  human  health. 

•  more  quantitative  computational  results   in   unanticipated  areas. 

•  more  explosive  groxvth  of  communications  as  a   component   of  the 
computational  science  and  engineering  paradigm;  and,   importantly, 

•  continued  progress  in  the  tools  and  methods  for  developing  code  that 
is  both  portable  and  yet  takes  advantage  of  unique  parallel 
architectures. 

These  advances   will   not  automatically  become  available  to  American 
researchers.     To  position  the  U.S.  academic  community  to  participate  in   the 
exciting  research  possibilities  enabled  by  these  developments,  the  Task  Force 
has  the  follozving  recommendations  leading  to  a  restructured  Centers 
program. 

In  order  to  maintain  world  leadership  in  computational  science  and 
engineering,  NSF  should  continue  to  maintain  a  strong,  viable  Advanced 
Scientific  Computing  Centers  program,   whose   mission   is: 

•  providing  access   to  high-end  computing  infrastructure  for  the 
academic   scientific   and   engineering   community; 

•  partnering  with  universities,  states,  and  industry  to  facilitate  and 
enhance  that  access; 

•  supporting  the  effective  use  of  such  infrastructure  through  training, 
consulting,  and  related  support  services; 

•  being  a  vigorous  early  user  of  experimental  and  emerging  high 
performance   technologies   that  offer  high  potential  for  advancing 
computational   science  and   engineering; 

•  facilitating  the  development  of  the  intellectual  capital  required  to 
maintain    world    leadership. 


18 


NSF  should  abSiire   that   the  Centers  program  provides   national    "Leading-edge 
>ites"  that  have  a  balanced  set  of  high-end  hardware  capabilities,  coupled  with 
appropriate  staff  and  software,   needed  for  continued  rapid  advancement  in 
computational   science   and   engineering. 

NSF,   through  its  Centers  program,  should  assure  that  each  leading-edge  site  is 
partnered   with  experimental  facilities  at   universities,   NSF  research  centers, 
and/or   national  and   regional  high  performance  computing     centers. 
Appropriate  funding  should  be  provided  for   the  partnership  sites. 

NSF  should  announce  a  new  competition  of  the  High  Performance 
Computing  Centers  program  that  would  permit  funding  of  selected  sites  for  a 
period  of  five  years.     If  regular  reviews  of  the  Program  and   the  selected   sites 
are  favorable,  it  should  be  possible  to  extend  initial  awards  for  an  additional 
five   years   without  a  full  competition. 

The  Centers  program  should  continue  to  support  need-based  research  in 
support  of  the  program's  mission,  but  should  not  provide  direct  support  for 
independent    research. 

NSF  should  increase  the  involvement  of  NSF's  directorates  in  the  process  of 
allocating  service  units  at  the  Centers. 

NSF  should  provide  leadership  in   working   toward  the  development  of 
interagency  plans  for  deploying  balanced  systems  at  the  apex  of  the 
computational  pyramid  and  ensuring  access  to  these  systems  for  academic 
researchers. 

National  Science  Board  Actions 

In  October  of  1994,  the  National  Science  Board  (NSB)  approved  a  two  year 
"extension  of  the  original  Centers  program  (through  FY  1997)  in  order  for  the 
Task  force  to  complete  its  work  and  to  allow  time  for  a  new  competition,  if 
NSF  recommended  one  and  the  NSB  agreed. 

The  Task  Force  report  was  presented  to  the  NSF  Director  on  September  15, 
1995  and  to  the  NSB  at  the  October  1995   meeting  on  the  Committee  on  Plans 
and  Policy.  The  NSF  recommendations  to  the  NSB  were  scheduled  for 
presentation  at  the  November  meeting,  but  were  postponed  to  December  by 
the  first  government  shutdown.  At  their  December  meeting  the  NSB 
approved  a  new  competition  for  a  program  called  Partnerships  in  Advanced 
Computational  Infrastructure  (PACI). 

The  Program  solicitation  for  the  Partnerships  meeting  was  released  on  the 
World  Wide  Web  (WWW)  on  December  15,  1995,    immediately  prior  to  the 
second  FY96  shutdown.  Since  that  time  a  printed  solicitation,  a  dear  colleague 
letter  and  a  series  of  informational  meetings  have  been  held  to  insure  that 
the  community  was  aware  of  the  program  and  had  all  the  required 
information  to  propose  to  participate.  In  addition,  a  set  of  frequently  asked 
questions  and  their  answers  has  been  posted  on  the  WWW  and  kept 
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continuously  updated.  The  complete  program  solicitation  is  appended  to  this 
testimony. 

Partnerships  program  competition 

Mission 

As  part  of  its  strategic  plan,  NSF  in  a  Changing  World,  a  key  NSF  goal  is  to 
enable  the  United  States  to  uphold  a  position  of  world  leadership  in  science 
and  engineering  research  and  education.    In  order  to  maintain  world 
leadership  in  computational  science  and  engineering,  NSF  intends  to  create 
an  adyanced  national  computational  infrastructure  whose  overall  mission  is 
to: 

•  provide,  facilitate,  and  enhance  access  to  high  performance  computational 
infrastructure  for  the  U.  S.  academic,  scientific,  and  engineering 
communities  by  partnering  with  universities,  states,  and  the  private 
sector; 

•  promote  vigorous  early  use  of  experimental  and  emerging  high 
performance  computational  and  associated  communications  technologies 
that  offer  high  potential  for  advancing  science  and  engineering; 

•  enable  the  effective  use  of  such  infrastructure  and  technologies  through 
education,  training,  consulting,  and  related  support  services,  including 
appropriate  software  development,  experimentation,  and  support; 

•  foster  interdisciplinary  research  in  science  and  engineering; 

•  facilitate  the  development  of  the  intellectual  capital  required  to  maintain 
world  leadership  in  computational  science  and  engineering;  and 

•  broaden  the  base  for  the  nation's  advanced  computational  and 
communications  infrastructure. 

Solicitation 

The  program  solicitation  for  the  Partnerships  for  Advanced  Computational 
Infrastructure  program,  builds  on  and  replaces  the  current  NSF 
Supercomputer  Centers  program,  and  focuses  on  taking  advantage  of  newly 
emerging  opportunities  in  high  performance  computing  and 
communications.   The  new  program  will  provide  flexibility,  both  to  adapt  to 
rapidly  evolving  circumstances  and  to  meet  the  need  for  high-end 
computation,  in  order  to  enable  continued  world  leadership  in 
computational  science  and  engineering.   The  program  will  provide  access  for 
researchers  to  high  performance  computing  systems  as  its  core,  with 
associated  highly  trained  staff  and  researchers  necessary  to  develop  and 
optimize  their  use.    The  emergence  of  scalable  parallel  systems,  high 
performance  networking  and  high  bandwidth,  large  capacity  mass  storage 
systems  creates  the  opportunity  for  a  national  infrastructure  consisting  of  a 
number  of  geographically  distributed  sites  strongly  coupled  to  high-end 
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computational  resources  and  to  each  other  \'ia  high-speeci  communication 
networks. 

NSF  envisions  an  Advanced  Computational  Infrastructure  consisting  of  one 
or  more  leading-edge  sites  together  with  cooperating  partners.   Leading-edge 
sites  are  expected  to  maintain  high-end  hardware  systems  that  are  one  to  two 
orders  of  magnitude  more  capable  than  those  typically  available  at  a  major 
research  university.   These  systems  should  be  balanced  in  terms  of  processor 
^peed,  memorv,  and  storage  systems,  and  should  be  accompanied  by 
appropriate  staff,  software  and  high  speed  communications  capability.   The 
partners  will,  in  the  aggregate,  complete  the  overall  infrastructure  by,  among 
other  things,  (a)  facilitating  research  and  experimentation  with  new  hardware 
and  software,  including  appropriate  support  technologies  such  as 
visualization  and  mass  storage,  (b)  providing  scalable  resources  for 
applications  and  applications  development  that  can  be  best  done  on  mid-level 
systems,  (c)  providing  access  to  unique  experimental  systems  and  facilities, 
and  (d)  promoting  education  and  training. 

Smce  the  Partnerships  for  Advanced  Computational  Infrastructure  Program 
will  primarily  support  academic  research,  it  is  expected  that  proposals  will 
come  from,  and  the  partnership's  cooperative  agreement  will  be  with,  a  U.  S. 
academic  institution.    However,  a  successful  proposal  must  involve  multiple 
partners  who  can  be,  but  are  not  limited  to: 

•  universities,  including  research  groups  within  universities; 

•  NSF-funded  Centers  and  facilities,  such  as.  Science  and  Technology 
Centers,  Supercomputer  Centers,  Engineering  Research  Centers,  and 
Industry-University  Cooperative  Research  Centers; 

•  research  and  educational  consortia,  organizations,  and  groups; 

•  regional  and  state-supported  high-performance  computing  centers; 

•  private  sector  organizations;  and 

•  federal  laboratories. 

A  full  description  of  organizations  and  individuals  who  are  eligible  to  submit 
proposals  to  NSF  and  the  conditions  under  which  they  may  compete,  are 

given  in  the  NSF  Grant  Proposal  Guide  (GPG)  (NSF  95-27). 

Review  Process 

Overview 

The  review  process  will  involve  three  stages;  preproposals,  full  proposals  and 
site  visits.   The  preproposals,  due  April  15,  1996,  will  be  evaluated  to 
determine  who  will  be  encouraged  to  submit  a  full  proposal.    The  intent  of 
the  preproposal  round  is  to  give  the  community  an  opportunity  to  put  forth 
ideas,  to  provide  proposers  with  information  that  will  assist  in  developing  a 
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stronger  proposal,  and  to  identity  at  an  early  stage,  preproposals  that  are  not 
expected  to  be  competitive.   Feedback  will  be  provided  to  all  preproposal 
submitters;  they  may  submit  a  full  proposal  regardless  of  the  opinions 
expressed  by  the  reviewers. 

Full  proposals  are  due  September  1,  1996.   Both  preproposals  and  full 
proposals  will  be  evaluated  by  external  panels.   In  late  fall,  1996, 
recommended  proposals  will  be  subjected  to  a  site  visit  (but  not  necessarily  all 
sites  in  a  given  partnership)  in  order  to  clarify  issues  raised  during  the  panel 
review,  and  to  explore  additional  matters  as  needed.    A  summary  panel, 
made  up  of  the  chairs  of  the  site  visit  teams  and  the  chair  of  the  full  proposal 
review  panel,  will  meet  after  the  site  visits. 

Criteria 

As  with  all  proposals  to  NSF,  these  proposals  will  be  subject  to  the  four 
standard  review  criteria,  i.e.   the  quality  of  the  proposed  scientific  effort, 
competence  of  the  investigators,  relevance  of  the  research,  and  impact  on  the 
infrastructure  of  science  and  engineering.    However,  for  this  particular 
solicitation,  impact  on  infrastructure  and  competence  of  investigators  will  be 
especially  important.   Thus  both  preproposals  and  final  proposals  for  the 
Partnerships  for  Advanced  Computational  Infrastructure  program  will  be 
subject  to  four,  more  specific,  criteria  dealing  with,  the  effectiveness  of  the 
overall  partnership  in  addressing  the  program's  mission,  the  quality  of  the 
individual  partners,  the  effectiveness  of  the  proposed  management,  and  the 
degree  of  financial  leverage.    While  these  four,  more  specific,  criteria  just 
mentioned  are  listed  in  priority  order  (with  the  first  being  the  most 
important),  all  will  be  considered  by  the  panels  in  arriving  at 
recommendations.    Finally,  the  summary  panel  will  be  asked  to  advise  NSF 
on  what  each  proposed  partnership  can  contribute  to  the  technical  diversity 
and  balance  of  the  program  as  a  whole. 

Management 

The  review  process  will  be  managed  by  the  Division  of  Advanced  Scientific 
Computing  (ASC).    In  order  to  assure  that  all  NSF  directorates  are  involved 
in  the  process,  an  NSF  internal  PACI  Review  Committee  will  be  established 
chaired  by  the  Centers  Program  Director  in  ASC.   Each  directorate  will  be 
invited  to  appoint  one  or  more  representatives  to  the  committee.    Ideally, 
there  should  be  strong  overlap  between  the  membership  of  the  High 
Performance  Computing  and  Communications  Coordinating  Committee  and 
the  Review  Committee.    The  Review  Committee  will  have  the  following 
responsibilities: 

•  Nominate  panelists  and  site  visitors  for  the  review  processes  outlined 
below. 

•  Participate  in  review  panels,  including  serving  as  moderators  of  possible 
hubpanels. 


10 


22 


•  Represent  their  directorate  on  site  visits  to  candidate  partnerships. 

•  Pro\ide  a  knowledgeable  link  between  their  directorate  and  the  process. 

Structure  of  Proposal  Review  Panels 

Panelists  will  be  chosen  from  a  variety  of  sectors:  academia,  the  private  sector, 
national  laboratories,  and  other  government  agencies.    Since  the  academic 
community  will  be  the  major  focus  of  the  PACI  Program,  subject  to  possible 
conflict  of  interests,  a  majority  of  reviewers  will  be  chosen  from  this  sector. 
As  described  above,  recommendations  from  all  NSF  directorates  for  possible 
panelists  will  be  solicited  through  the  PACI  Review  Committee.    Reviewers 
will  be  chosen  from  the  computational  science  and  engineering  and 
computer  science  and  engineering  communities  whose  areas  of  expertise 
represent  those  of  users  of  the  infrastructure,  enablers  of  the  underlying 
technologies,  and  managers  of  large  facilities,  and  encompass  both  research 
and  educational  needs  for  the  PACI  program. 

Preproposal  panel  procedure 

Written  reviews  of  each  preproposal  will  be  provided  by  at  least  3  readers,  and 
a  summary  of  the  subpanel  discussion  of  each  preproposal  will  be  recorded  on 
a  panel  review  form  which  highlights  the  major  review  criteria.    The  task  of 
the  subpanels  will  be  to  recommend  whether  a  full  proposal  is  to  be 
encouraged  or  not.   A  target  number  of  proposals  to  be  encouraged  will  be 
suggested  to  the  panels.    All  reviews  and  the  panel  summary  will  be  provided 
to  the  PI. 

No  PI  or  named  collaborator  will  be  chosen  to  be  a  reviewer  (panelist)  due  to 
disqualifying  conflict  of  interest.    Reviewers  from  the  same  institution  may 
have  to  be  used  due  to  the  large  number  of  partners  expected  in  the 
submissions,  but  an  attempt  will  be  made  to  segregate  these  into  subpanels 
considering  preproposals  not  involving  their  institution;  if  this  is  not 
possible,  the  reviewer  will  be  absent  from  the  discussion  of  any  preproposal 
involving  his/her  institution. 

Full  proposal  panel  procedure 

The  panelists  will  be  chosen  from  the  original  subpanels  reviewing  the 
preproposals,  from  the  pool  of  submitters  of  preproposals  not  submitting  full 
proposals,  and  from  the  community  at  large.    While  there  will  be  some 
overlap  between  the  panels  reviewing  the  preproposals  and  the  final 
proposals,  the  panelists  will  be  instructed  that  any  recommendations  based  on 
the  preproposal  review  were  not  intended  to  be  binding  on  the  proposer  and 
should  not  be  considered  in  reviewing  the  final  proposal.    Preproposal 
reviews  and  recommendations  will  not  be  available  to  the  panelists. 

Each  reviewer  will  be  provided  with  all  of  the  proposals  and  a  specially 
prepared  review  form  which  will  reflect  the  major  review  criteria.    Each 
proposal  will  have  at  least  3  written  reviews  prepared  by  reviewers  prior  to 
the  panel  discussion;  each  panelist  will  be  assigned  approximately  4  of  the 
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proposals  tor  which  he/she  must  prepare  written  reviews.    These  written 
reviews  will  include  ratings  categorizing  the  assigned  proposals  into  the  top 
25"/u,  the  next  IS"";.,  and  the  remainder.    All  proposals  will  then  be  discussed  in 
the  panel.    The  results  of  the  panel  discussions  will  be  recommendations 
which  rate  the  proposals  as  "definitely  site  visit",  "possibly  site  visit",  and  "do 
not  consider  further".    All  reviews  and  a  panel  summary  will  be  provided  to 
the  PI. 

It  is  expected  that  conflicts  will  be  less  than  in  the  preproposal  stage,  and  any 
that  arise  will  be  handled  in  a  similar  manner. 

Site  visit  procedure 

NSF  staff  will  review  the  recommendations  for  site  visits.    In  order  that  the 
portfolio  for  site  visits  assures  consideration  of  a  balanced  program,  NSF  may 
possibly  augment  the  "definitely  visit"  group  with  some  from  the  "possibly 
visit"  group. 

All  proposals  recommended  will  be  reviewed  by  a  team  of  site  visitors 
consisting  of  previous  panelists  augmented  by  other  reviewers  as  appropriate. 
Partnerships  to  be  visited  will  be  provided  with  a  set  of  questions  prior  to  the 
visit.    NSF  staff  members,  including  members  of  the  PACI  Review 
Committee  described  above,  will  be  invited  to  participate  in  all  site  visits  as 
observers.   Each  site  visit  team  will  have  a  chair  designated  by  NSF.   Each 
visitor  will  go  on  at  least  two  site  visits  (preferably  more)  in  order  to  have  an 
overview  of  the  possible  varieties  of  partnerships  available.    The 
recommendations  of  the  team  will  be  to  either  fund,  fund  if  possible,  or  not 
fund  the  partnership. 

Upon  completion  of  all  site  visits,  a  panel  of  the  site  visit  chairs,  chaired  by 
the  chair  of  the  proposal  review  panel,  will  meet  at  NSF  to  review  the  overall 
recommendations  made  by  the  site  visit  teams,  and  to  provide  NSF  with 
several  options  (e.g.,  one  large  partnership,  or  several  smaller  partnerships) 
which  meet  the  overall  mission  of  the  PACI  Program. 

Management 

Active  management  of  the  program  by  the  Foundation  is  essential,  since  the 
new  program  has  been  designed  to  provide  a  national  infrastructure  that  is 
built  on  one  or  more  multi-institutional  partnerships.    Because  of  the 
complexity  and  distributed  nature  of  the  new  program,  both  external  and 
internal  management  issues  must  be  considered.    The  management  model 
draws  on  experience  gained  with  a  number  of  other  multi-institution  NSF 
programs,  but  management  of  this  program  is  still  expected  to  offer  unique 
challenges. 

Each  partnership  will  work  with  NSF  to  develop  criteria  by  which  it  can  be 
evaluated.   These  criteria  will  be  used  in  all  formal  reviews.   Each  partnership 
will  submit  an  annual  program  plan  which  summarizes  its  progress  and 
describes  activities  and  budgets  for  the  coming  year.   This  may  include 
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proposed  changes  in  membership,  research,  service,  networking,  computing 
bvstems,  or  funding  changes  for  its  members.   These  plans  will  be 
indi\-iduaily  reviewed  by  an  external  program  review  panel  appointed  by 
NSF  which  will  provide  recommendations  to  NSF  regarding  funding  levels, 
addition  of  or  deletion  of  partners,  new  opportunities,  etc.    Although  they 
will  not  be  members  of  the  panel,  it  is  expected  that  representatives  of  all 
partners  in  each  partnership  will  be  at  these  reviews  and  that  their  locations 
may  vary  between  partnership  sites  and  NSF. 

To  provide  flexibility  in  meeting  special  needs,  funding  can  be  provided  for 
partnerships  having  a  membership  that  may  change  over  the  duration  of  the 
ceioperative  agreement;  thus  initial  proposals  involving  partners  for  a  shorter 
period  are  encouraged,  where  appropriate.    Furthermore  the  partnership  can 
propose,  at  any  time,  a  change  in  the  number  or  composition  of  its  partner 
sites.    Normally  such  proposals  will  be  part  of  the  annual  program  plan 
review  process  and  would  be  considered  along  with  other  substantive  issues. 
Possible  justifications  for  such  a  change  might  be  the  occurrence  of  an 
unusual  opportunity,  completion  of  a  partner's  task,  changes  or  relocation  of 
a  critical  person,  etc. 

More  frequent  informal  monitoring  will  be  carried  out  by  NSF  staff.   Each 
partnership  will  be  required  to  contribute  to  a  database  relating  to  usage, 
users,  training,  outreach,  education,  industrial  interactions,  and  other 
significant  data  relating  to  the  operation  of  the  program.    NSF  will  also 
require  periodic  meetings  of  the  Pi's  and  co  Pi's  of  the  partnerships  for  overall 
program  coordination.    This  will  permit  the  Foundation  to  better  understand 
and  monitor  both  the  operation  of  a  partner  within  the  partnership,  the 
partnership  itself,  and  the  program  as  a  whole. 

A  single  institution,  through  the  PI,  is  expected  to  be  responsible  for  overall 
management  and  leadership  of  the  partnership  as  a  whole,  and  each  partner 
site  will  have  a  local  director.    Partnerships  will  have  a  policy  and  operations 
management  committee  that  will  meet  regularly.    Its  membership  will 
consist  of  a  senior  representative  from  each  site.    NSF  will  host  an  annual 
meeting  of  representatives  from  all  the  partnerships  to  discuss  program  wide 
coordination. 

Every  partnership  will  be  encouraged  to  appoint  an  independent  external 
advisory  committee  which  will  provide  oversight  and  guidance  necessary  for 
the  management  of  the  partnership.    This  committee  will  include  users  of  the 
partnership  resources  and  others  who  are  in  a  position  to  evaluate  the 
activity  and  provide  feedback  to  its  management.   This  group  will  be  selected 
in  consultation  with  program  management  staff  and  will  meet  at  least  once  a 
year. 
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The  Transition 

The  transition  from  the  old  Centers  Program  to  the  new  Partnerships 
Program  will  take  place  beginning  immediately  after  Board  approval  of  the 
new  partnerships. 

Phase  in  of  a  new  partnership  is  expected  to  occur  over  a  one  year  period. 
Partnership  plans  and  proposed  budgets  must  be  prepared  with  enough 
flexibility  to  allow  the  phasing  in  of  partnership  resources  during  the  first 
vear,  as  negotiated  with  NSF. 

It  is  critical  that  this  transition  be  conducted  smoothly  to  ensure  that  there  is 
continuity  in  the  availability  of  high  performance  computing  resources  to 
support  the  research  of  the  academic  science  and  engineering  community. 

The  final  transition  plan  will  be  designed  in  accordance  with  the  principles 
below: 

•  minimum  disruption  for  the  scientific  research  community; 

•  deferral   of  all  FY97  Supercomputer  Centers  program  expenditures 
possible,  until  awards  for  the  new  program  are  announced; 

•  rapid  phase  out  of  old  participants;  rapid  phase  in  of  new  partners; 

•  immediate  and  active  NSF  involvement  in  the  transition  process. 

Summary 

At  the  conclusion  of  this  exercise,  NSF  will  continue  world  class  support  of 
computational  science  and  engineering  for  at  least  another  5  years.  During  the 
fourth  year  of  the  program  an  evaluation  will  determine  whether  to  continue 
without  recompetition  for  another  five  years  at  which  the  program  will  be 
"sunset".  Before  this  occurs  another  review  will  determine  the  future  for 
continued  NSF  support  of    the  infrastructure. 
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Mr.  SCHIFF.  Well,  Dr.  Hayes,  you  got  quite  an  introduction  from 
the  first  witness,  so  with  that  I  will  call  upon  you,  please. 

STATEMENT  OF  DR.  EDWARD  HAYES,  CHAIRMAN,  REPORT  ON 
THE  TASK  FORCE  ON  THE  FUTURE  OF  NSF 
SUPERCOMPUTING  CENTERS  PROGRAM  AND  VICE  PRESI- 
DENT FOR  RESEARCH,  OHIO  STATE  UNIVERSITY,  COLUM- 
BUS, OHIO 

Dr.  Hayes.  Thank  you,  Mr.  Chairman. 

I  would  also  like  to  add  my  thanks  to  you  and  to  the  Committee 
for  their  strong  interest  in  the  Supercomputing  Centers  Program 
and  their  bipartisan  support  over  the  years  for  the  science  pro- 
grams funded  by  the  Federal  Government. 

In  addition  to  being  the  chair  of  the  Task  Force,  my  da5rtime  job 
is  vice  president  for  research  at  the  Ohio  State  University;  and  my 
research  interests  are  in  the  area  of  computational  chemistry. 

I  am  very  pleased  to  have  this  opportunity  to  talk  with  you  today 
about  the  new  partnerships  for  Advanced  Computational  Infra- 
structure, a  Program  that  was  recently  announced  and  that  Paul 
has  been  talking  about. 

My  objective  is  to  give  you  a  context  as  well  as  a  brief  overview 
of  the  Task  Force  Report.  A  key  message  that  I  would  like  to  leave 
with  the  Committee  today  is  how  pleased  the  Task  Force  is  with 
the  process  that  NSF  has  followed  in  setting  the  directions  for  the 
future  of  its  highly  successful  Supercomputing  Centers  Program. 

The  members  of  the  Task  Force  are  listed  in  my  prepared  re- 
marks. This  is  a  group  with  exceptional  talent  and  breadth  of  expe- 
rience. It  made  the  job  of  the  chair  quite  enjoyable,  as  well  as  chal- 
lenging. 

All  are  knowledgeable  about  one  or  more  aspects  of  high-perform- 
ance computing.  They  are  all  independent-minded  and  committed 
to  doing  their  work  in  a  professional  way.  They  are  pleased,  and 
we  were  pleased  to  be  able  to  report  the  unanimous  view  of  what 
steps  should  be  recommended  to  the  National  Science  Foundation 
on  the  future  of  the  program. 

In  essence,  we  found  that  the  Supercomputing  Centers  Program 
has  been  a  critical  component  of  the  strong  U.S.  position  in  science 
and  technology,  and  we  strongly  recommended  that  it  be  continued 
but  with  important  changes. 

With  the  issuance  of  the  new  Partnerships  Program  announce- 
ment, the  National  Science  Foundation  initiated  the  important  pro- 
posal development  stage.  At  this  stage,  various  groups  of  univer- 
sities and  other  potential  partners  are  putting  their  best  ideas  to- 
gether to  see  if  they  will  be  able  to  convince  their  peers  that  they 
have  the  best  plans  to  enable  the  future  of  computational  science 
and  engineering. 

While  our  Task  Force  set  the  stage  for  the  final  program  an- 
nouncement, and  the  program  announcement  set  the  stage  for 
these  proposal  development  efforts,  there  is  still  significant  room 
for  creativity  on  the  part  of  the  Partnership  Centers. 

The  Task  Force  clearly  understood  the  importance  of  not  tying 
the  hands  of  creative  people  at  the  proposal  development  stage. 
From  this  perspective,  we  are  pleased  with  the  clarity  of  focus  ex- 
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emplified  in  the  program  announcement,  as  well  as  the  appro- 
priateness of  the  selection  criteria. 

The  competitive  responses  often  bring  out  important  new  ideas, 
insights,  and  commitments  that  cannot  be  accurately  forecast  in 
advance.  The  final  stage  of  the  process  will  begin  after  the  propos- 
als are  received  and  the  merit  review  begins. 

The  Task  Force  has  a  very  high  level  of  confidence  in  Paul 
Young,  the  assistant  director,  who  has  line  management  respon- 
sibility for  this  Program.  He  and  his  staff  are  most  knowledgeable 
in  this  area  and  have  a  great  track  record  for  running  fair  and  ef- 
fective review  processes. 

At  this  point  I  want  to  make  just  a  few  brief  comments  from  my 
statement  that  focus  on  just  a  couple  of  aspects  of  the  program. 

The  Task  Force  heard  from  many  knowledgeable  individuals 
from  industry,  academia,  government,  and  government  labora- 
tories, NSF  assistant  directors,  academic  researchers,  representa- 
tives of  other  federal  agencies,  center  directors,  as  well  as  members 
of  the  centers  allocation  committees.  We  talked  with  them  all. 

Considerable  input  was  received  from  many  thoughtful  people. 
This  input,  as  well  as  our  survey  to  which  there  were  500  replies, 
was  essential.  It  extended  the  knowledge  base  of  the  individual 
members  of  the  Task  Force,  and  informed  us  about  the  key  issues 
surrounding  the  Centers  Program. 

This  input  also  gave  us  an  excellent  sense  of  the  important  role 
that  the  Centers  Program  has  played  in  making  computational 
science  and  engineering  a  significant  contributor  to  the  long-term 
progress  of  fundamental  research  and,  importantly,  in  providing 
the  infrastructure  needed  for  education  and  training  of  future  gen- 
erations of  world  leaders  in  science  and  technology. 

The  recommendations  of  the  Task  Force  follow  from  our  vision  of 
the  future  of  high-performance  computing.  This  vision  has  several 
key  points.  I  want  to  mention  just  one  of  these — there  are  seven 
in  total — this  afternoon.  The  others  are  listed  in  my  prepared  re- 
marks. 

A  new  element  of  our  vision  is  a  strong  coupling  of  selected  re- 
search centers  and  university  laboratories  with  the  leading  edge 
sites  that  have  the  highest  end  computational  systems. 

We  believe  that  such  coupling  has  great  potential  for  the  future 
in  terms  of  an  enhanced  program  flexibility,  creativity,  and  effi- 
ciency. 

With  advances  in  high-speed  communications,  this  approach  will 
also  provide  the  infrastructure  needed  to  support  creative  experi- 
ments with  distributed  computing. 

In  my  prepared  remarks  I  have  commented  on  each  of  the  Task 
Force  recommendations.  Two  of  these  recommendations  are  central 
to  the  new  Partnerships  Program. 

In  our  second  recommendation  we  noted  that  NSF  needs  to  sup- 
port a  few  leading  edge  sites,  sites  that  have  a  balanced  set  of 
high-end  hardware  capabilities,  coupled  with  appropriate  staff  and 
software.  Raw  processor  speed  or  compute-  power  is  not  sufficient 
to  achieve  the  balanced  high-performance  systems  needed  for  para- 
digm-shifting research. 

In  our  third  recommendation  we  focused  on  partnerships.  Part- 
nerships will  be  important  for  the  future  of  the  program.  With  stra- 


28 

tegic  coupling  between  leading-edge  sites  and  research  centers  and 
university  laboratories,  it  is  a  significant  new  change. 

These  partnership  sites  will  provide  better  coupling  to  computer 
scientists  developing  new  tools  and  software.  This  coupling  will  be 
particularly  important  when  their  research  on  mid-range  systems 
reaches  the  point  when  it  needs  to  examine  scalability  issues  that 
become  manifest  only  on  the  leading  edge  systems.  But  the  impact 
of  this  research  will  extend  beyond  computer  science. 

The  partnership  sites  will  also  provide  more  cost-effective  and  di- 
verse platforms  for  early  design  and  testing  of  applications  soft- 
ware prior  to  the  stage  where  access  to  a  leading-edge  system  is 
required. 

Although  we  began  with  the  work  of  earlier  reports,  we  did  not 
accept  the  conclusions  or  the  recommendations  uncritically.  We  car- 
ried out  our  own  analysis  particularly  of  the  merit  review  process 
for  allocating  resources  at  the  current  centers,  an  analysis  of  the 
educational  benefits  of  the  program,  and  we  also  sought  significant 
input  from  the  community,  including  several  key  NSF  advisory 
committees  particularly  on  the  benefits  of  the  current  program. 

In  preparing  for  this  hearing  I  was  able  to  talk  to  most  of  the 
members  of  the  Task  Force  to  confirm  their  enthusiasm  for  the 
manner  in  which  NSF  management  is  moving  forward  with  the 
program  announcement  and  the  receipt  of  proposals.  All  indicated 
that  they  are  pleased  with  the  progress  that  has  been  made  up  to 
this  point. 

Thank  you,  Mr.  Chairman. 

[The  prepared  statement  and  attachments  of  Dr.  Hayes  follow:] 

Statement  of  Dr.  Edward  F.  Hayes 

Vice  President  for  Research 

The  Ohio  State  University 

AND 

Chair,  Task  Force  on  the  NSF  Supercomputer  Program 

Mr.  Chairman  and  Members  of  the  Subcommittee: 

I  am  very  pleased  to  have  this  opportunity  to  talk  with  you  about  the  New  Part- 
nerships for  Advanced  Computational  Infrastructure  program  that  was  recently  an- 
nounced by  the  National  Science  Foundation. 

One  of  the  relevant  reports  that  guided  the  National  Science  Foundation  in  the 
design  of  this  new  program  was  the  Report  of  the  Task  Force  on  the  Future  of  the 
NSF  Supercomputer  Program.  It  was  my  honor  to  be  asked  by  the  Director  of  the 
National  Science  Foundation  to  chair  this  Task  Force. 

My  objective  today  is  to  give  you  a  context  as  well  as  a  brief  overview  of  the  Task 
Force  Report.  A  brief  overview  is  not  an  easy  task  because  our  report  deals  with 
many  significant  questions  and  issues.  A  key  feature  of  our  report  is  the  extent  to 
which  we  dealt  with  the  tough  issues  and  seriously  considered  alternatives. 

A  key  message  that  I  would  like  leave  with  the  committee  is  how  pleased  the 
Task  Force  has  been  with  the  process  that  NSF  has  followed  in  setting  the  direc- 
tions for  the  future  of  its  highly  successful  Supercomputer  Centers  Program. 

The  Task  Force  was  appointed  by  the  Director  of  the  Foundation,  Dr.  Neal  Lane, 
and  asked  to  provide  him  with  recommendations  on  the  future  of  the  Supercomputer 
Centers  Program  and  to  report  our  findings  as  soon  as  possible.  The  work  of  the 
Task  Force  began  in  January  of  1995  and  culminated  in  our  final  report  to  the  Di- 
rector last  September. 

The  members  of  the  Task  Force  are  listed  below: 
•  Dr.  Arden  L.  Bement,  Jr.,  Basil  S.  Turner  Distinguished  Professor  Of  Engineering, 
Purdue  University 
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•  Chairman — Dr.  Edward  F.  Hayes,  Vice  President  for  Research,  The  Ohio  State 

University 

•  Dr.  John  Hennessy,  Chair  Computer  Science,  Stanford  University 

•  Dr.  John  Ingram,  Schlumberger  Research  Fellow,  Schlumberger,  Austin 

•  Dr.  Peter  A.  KoUman,  Professor  of  Chemistry  and  Pharmaceutical  Chemistry,  Uni- 

versity of  California — San  Francisco 

•  Dr.  Mary  K.  Vernon,  Professor  of  Computer  Science,  University  of  Wisconsin 

•  Dr.  Andrew  B.  White,  Jr.,  Director  of  Advanced  Computing  Laboratory,  Los  Ala- 

mos National  Laboratory 

•  Dr.  William  A.  Wulf,  AT&T  Professor  of  Engineering  and  Applied  Science,  Univer- 

sity of  Virginia 
Ex-officio  Non-Voting  members: 

•  Dr.  Robert  G.  Voigt,  NSF,  Acting  HPCC  Coordinator 

•  Dr.  Paul  R  Young,  NSF,  Assistant  Director,  CISE 

•  Dr.  Nathaniel  G.  Pitts,  NSF,  Director/OSTI 

There  is  considerable  talent  and  experience  in  this  group.  Some  of  these  names 
may  be  famihar.  All  are  knowledgeable  about  one  or  more  aspects  of  high  perform- 
ance computing.  They  are  all  independent  minded  and  committed  to  doing  their 
work  in  a  professional  way.  We  did  not  start  out  with  a  common  view  of  what  rec- 
ommendations we  might  make  to  the  National  Science  Foundation.  In  fact,  we  had 
many  discussions  in  which  one  or  more  members  of  the  group  were  observed  in  var- 
ious stages  of  testiness.  However,  I  believe  that  we  were  all  pleased  to  be  able  to 
report  that  after  a  great  deal  of  effort  on  everyone's  part  we  did  come  to  a  unani- 
mous view  of  what  steps  should  be  recommended.  In  essence,  we  found  that  the 
Supercomputer  Centers  Program  has  been  a  critical  component  of  the  strong  U.S. 
position  in  science  and  technology  and  we  strongly  recommended  that  it  be  contin- 
ued, but  with  important  changes. 

After  receiving  our  report,  the  Director  of  the  Foundation  initiated  the  second 
stage  of  the  process.  During  this  stage  he  reviewed  the  report  with  key  staff  within 
the  agency  and  formulated  an  action  plan  to  be  discussed  with  the  National  Science 
Board  for  their  review  and  approval.  The  National  Science  Board  provides  a  very 
important  perspective  on  major  Foundation  programs  because  they  represent  a  very 
broad  cross  section  of  expertise  and  considerable  experience  with  policy  issues  and, 
importantly,  experience  in  setting  priorities  in  the  context  of  the  whole  NSF  and 
Federal  Research  Budgets.  At  the  October  1995  meeting  of  the  National  Science 
Board,  Arden  Bement,  another  member  of  the  Task  Force,  and  I  were  asked  to  pro- 
vide members  of  the  Board  with  an  overview  of  the  Task  Force  Report.  At  this  meet- 
ing the  Board  also  heard  from  Fred  Brooks,  co-chair  of  the  NRC  report  on  the  HPCC 
Program,  and  Jerry  Ostriker,  an  eminent  computational  scientist  who  has  made 
many  significant  contributions  to  our  understanding  of  cosmology. 

Following  the  October  meeting  of  the  National  Science  Board,  the  third  stage  of 
the  review  process  began  as  the  NSF  Director  put  together  the  proposed  program 
announcement  for  the  new  partnerships  program.  The  Board  was  scheduled  to  dis- 
cuss and  act  upon  this  proposal  at  its  November  meeting,  but  strange  things  were 
happening  with  the  FY  1996  Budget  discussions  during  this  period — unfortunately 
the  NSF  was  shut  down  during  the  period  of  the  scheduled  November  Board  meet- 
ing. Ultimately,  the  NSF  was  able  to  stay  open  long  enough  for  the  Board  to  meet 
in  December — at  which  time  the  program  announcement  was  given  final  approval 
and  was  quickly  put  out  on  the  streets  just  before  the  government  shut  down  again. 

The  issuance  of  the  Program  Announcement  began  the  fourth  stage  in  the  proc- 
ess— the  important  proposal  development  stage  is  when  various  groups  of  univer- 
sities and  other  potential  partners  put  their  best  ideas  together  to  see  if  they  will 
be  able  to  convince  their  peers  that  they  have  the  best  plans  to  enable  the  future 
of  computational  science  and  engineering.  While  our  Task  Force  set  the  stage  for 
the  final  program  announcement  and  the  program  announcement  set  the  stage  for 
these  proposal  development  efforts,  there  is  still  significant  room  for  creativity  on 
the  part  of  the  partnership  centers.  The  Task  Force  clearly  understood  the  impor- 
tance of  not  tying  the  hands  of  creative  people  at  the  proposal  development  stage. 
From  this  perspective,  we  are  pleased  with  the  clarity  of  focus  exemplified  in  the 
program  announcement  as  well  as  the  appropriateness  of  the  selection  criteria.  The 
competitive  responses  at  this  stage  often  bring  out  important  new  ideas,  insights 
and  commitments  that  cannot  be  accurately  forecast  in  advance. 

The  final  stage  will  begin  after  proposals  are  received  and  the  merit  review  be- 
gins. The  Task  Force  has  a  very  high  level  of  confidence  in  Paul  Young,  the  Assist- 
ant Director,  who  has  line-management  responsibility  for  this  program.  He  and  his 
staff  are  most  knowledgeable  in  this  area  and  have  a  great  track  record  in  running 
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fair  and  effective  review  processes.  If  we  can  just  keep  the  agency  open,  they  will 
do  a  great  job  in  picking  the  winners. 

At  this  point,  I  want  to  return  to  the  report  of  the  Task  Force. 

The  current  NSF  Supercomputer  Centers  Program  was  initiated  in  the  1985-86 
time  period  with  the  formation  of  five  supercomputer  centers.  Each  one  had  a  vector 
supercomputer  that  was  considered  big  for  the  time  period.  Over  the  first  five  years 
of  the  program's  existence  these  centers  provided  vector  supercomputing  services  for 
the  research  community  and  training  for  the  many  researchers  who  lacked  experi- 
ence with  such  systems.  Prior  to  1985,  U.S.  academic  researchers  had  very  limited 
access  to  vector  supercomputers. 

At  the  end  of  the  first  five  years,  NSF  held  a  major  external  review  of  the  five 
Centers.  Four  of  the  five  Centers  were  approved  for  continuation  for  another  five 
years.  The  John  von  Neuman  Center  in  New  Jersey  was  not  renewed. 

In  the  second  five  years  of  the  Center  Program's  existence  several  significant 
changes  were  made: 

•  There  was  a  distinct  effort  to  expand  the  outreach  efforts  of  the  centers  and  to 

forge  closer  ties  with  industryy. 

•  The  four  remaining  centers — at  Cornell  University,  the  University  of  Illinois,  the 

Pittsburgh  Center  and  the  San  Diego  Center — formed  themselves  into  a 
MetaCenter  with  resources  sharable  on  a  national  scale  and  coordinated  plan- 
ning to  develop  and  not  duplicate  areas  of  special  expertise  within  the  program. 

•  Finally  there  was  a  move  to  change  the  computational  paradigm  as  a  new  genera- 

tion of  parallel  architecture  machines  started  hitting  the  market. 

So  it  was  after  ten  years  and  many  successes  that  our  Task  Force  was  called  to- 
gether to  advise  the  National  Science  Foundation  on  the  future  of  the  Centers  Pro- 
gram. 

In  arriving  at  our  conclusions,  we  benefited  significantly  from  two  earlier  reports 
that  I  want  to  comment  on  briefly  because  of  their  direct  relevance  to  our  own  work. 

From  Desktop  Teraflop:  Exploiting  the  U.S.  Lead  in  High  Performance  Computing, 
NSF  Blue  Ribbon  Panel  on  High  Performance  Computing,  August  1993 

Evolving  the  High  Performance  Computing  and  Communications  Initiative  to  Sup- 
port the  Nation's  Information  Infrastructure,  Computer  Science  and  Telecommuni- 
cations Board,  National  Research  Council,  1995 

Both  of  these  reports  speak  to  the  importance  of  exploiting  the  U.S.  lead  in  high 
performance  computing. 

The  Blue  Ribbon  Panel  emphasized  the  importance  of  maintaining  a  balance  in 
the  pyramid  of  computational  capability.  See  Attachment  1. 

The  NRC/HPCC  Committee  made  several  important  observations  central  to  the 
work  of  our  Task  Force. 

I  would  like  to  quote  a  few  sentences  from  the  executive  summary  of  the  HPCC 
report.  These  quotes  capture  for  me,  in  a  succinct  way,  the  importance  of  high  per- 
formance computing  for  the  nation. 

In  the  section  dealing  with  high  performance  we  read: 

"  Tiigh  performance' — which  involves  bringing  more  powerful  computing  and  com- 
munications technology  to  bear  on  a  problem — has  enabled  advances  on  several 
fronts.  High  performance  system.s,  for  example,  deliver  answers  sooner  to  complex 
problems  that  need  large  amounts  of  computing.  Timely  and  accurate  forecasting  of 
weather,  mapping  of  oil  reservoirs,  and  imaging  of  tumors  are  among  the  benefits 
encompassed  by  the  goals  of  the  HPCC  initiative." 

Then  continuing  on  in  the  next  paragraphh. 

"Information  technology  evolves  as  new  and  valuable  applications  are  found  for 
hardware  that  gets  steadily  more  powerful  and  cheaper.  To  benefit,  users  need  af- 
fordable hardware,  but  they  also  need  the  software  that  implements  the  new  appli- 
cations. Yet  learning  how  to  build  software  takes  many  years  of  experimentation. 
If  this  process  starts  only  when  the  hardware  has  already  become  cheap,  the  bene- 
fits to  users  will  be  delayed  by  years.  Research  needs  to  treat  todays  expensive 
equipment  as  a  time  machine,  learning  how  it  will  be  used  when  it  is  cheap  and 
widely  available,  as  surely  it  will  be  tomorrow." 

In  addition  to  the  insights  obtained  from  these  two  reports,  the  Task  Force  heard 
from  many  knowledgeable  individuals  from  industry,  academia,  government,  and 
government  laboratories.  NSF  Assistant  Directors,  Academic  Researchers,  Rep- 
resentatives of  other  Federal  Agencies,  Center  Directors  as  well  as  members  of  the 
centers  allocation  committees — we  talked  with  them  all.  Considerable  input  was  re- 
ceived from  many  thoughtful  people.  This  input  as  well  as  our  survey,  to  which  over 
500  replied,  was  essential.  It  extended  the  knowledge  base  of  the  individual  mem- 
bers of  the  Task  Force  and  informed  us  about  the  key  issues  surrounding  the  Cen- 
ters Program.  This  input  also  gave  us  an  excellent  sense  of  the  important  role  that 
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the  Centers  program  has  played  in  making  computational  science  and  engineering 
a  significant  contributor  to  the  long  term  progress  of  fundamental  research,  and,  im- 
portantly, in  providing  the  infrastructure  needed  for  the  education  and  training  of 
future  generations  of  world  leaders  in  science  and  technology. 

Today,  much  of  the  important  computational  science  and  engineering  is  being  car- 
ried out  on  workstations  and  mid-range  machines.  That  is  desirable.  It  is  one  of  the 
clear  signs  of  the  health  of  the  U.S.  high  performance  computing  program.  A  signifi- 
cant amount  of  the  applications  software  that  is  being  run  on  today's  workstations 
was  developed  during  the  early  days  of  the  NSF  Supercomputer  Centers  Program. 
The  systems  of  that  period  had  about  the  same  power  as  today's  workstations. 
These  earlier  "time  machines"  worked  just  the  way  people  predicted  that  they  would 
ten  years  ago. 

The  recommendations  of  the  Task  Force  follow  from  our  vision  for  the  future  of 
high  performance  computing.  This  vision  has  seven  key  points  that  I  summarize 
below. 
Future  Vision 

•  Computational  Science  &  Engineering  is  central  to  the  long  term  development  of 

fundamental  principles  and  understanding  of  complex  systems 

•  Continued  U.S.  leadership  in  computational  science  and  engineering  requires  ac- 

cess to  the  highest  end  computational  systems,  systems  that  are  balanced  with 
respect  to  processor  speed,  memory,  storage  and  software 

•  Coupling  of  selected  mid-range  systems  to  the  leading-edge  systems  has  great  po- 

tential in  the  future  in  terms  of  enhanced  program  flexibility,  creativity,  and 
efficiency 

•  Research  on  computing  tools,  algorithms  and  new  models  of  physical  systems  pays 

significant  dividends,  but  such  research  takes  time  and  commitment  particu- 
larly with  changes  in  the  underlying  computing  paradigm 

•  Unanticipated  results  are  often  the  most  important 

•  Research  trains  people  for  the  future  needs  of  the  nation 

•  SjTiergy  among  industry,  academia,  and  government 

Computational  science  and  engineering  is  central  to  the  long  term  development 
of  fundamental  principles  and  the  understanding  of  complex  systems.  Computa- 
tional science  brings  enhanced  synthesis  and  analysis.  It  is  the  source  of  new  in- 
sights that  facilitate  a  deeper  understanding  of  fundamental  issues  and,  impor- 
tantly, computational  science  is  a  full  partner  with  experimental  and  theoretical 
science  in  extending  the  analysis  of  experimental  results  that  cannot  be  fully  syn- 
thesized without  the  assistance  offered  by  sophisticated  computational  models. 

Continued  leadership  in  computational  science  and  engineering  requires  access  to 
the  highest  end  computational  systems.  Here  we  mean  systems  that  are  balanced 
with  respect  to  processor  speed,  numbers  of  processors,  memory,  storage,  software 
and  personnel.  These  systems  act  as  time  machines  for  academic  researchers  allow- 
ing them  to  gain  significant  lead  time  in  addressing  computational  problems  and  is- 
sues that  emerge  whenever  significant  new  levels  of  modeling  capability  emerge. 

A  new  element  in  our  vision  is  a  strong  coupling  of  selected  research  centers  and 
university  laboratories  with  the  leading-edge  sites  that  have  the  highest  end  com- 
putational systems.  We  believe  that  such  coupling  has  great  potential  for  the  future 
in  terms  of  enhanced  program  flexibility,  creativity,  and  efficiency.  With  advances 
in  high  speed  communications  this  approach  will  also  provide  the  infrastructure 
needed  to  support  creative  experiments  with  distributed  computing. 

The  fourth  element  in  our  vision  focuses  on  the  nature  and  time  scale  for  human 
creativity.  Research  on  computing  tools,  algorithms  and  new  models  of  physical  sys- 
tems pay  significant  dividends  but  such  research  takes  time  and  commitment  par- 
ticularly when  associated  with  changes  in  the  underlying  computing  paradigm. 

This  fourth  element  as  well  as  the  final  three  are  themes  that  are  also  dealt  with 
in  the  HPCC  report. 

Our  recommendations  follow  directly  from  this  vision. 
Recommendations 

•  Continuing  Need  for  the  Centers  Program 

•  Specific  infrastructure  Characteristics  for  Leading-Edge  Sites 

•  Partnering  for  a  More  effective  National  Infrastructure 

•  Competition  and  Evaluation 

•  Support  of  Research  at  the  Centers 

•  Allocation  Process  for  Computer  Service  Units 

•  NSF  Leadership  in  Interagency  Planning 

The  first  recommendation  is  that  NSF  should  continue  to  provide  access  to  the 
high-end.  This  infrastructure  is  central  to  maintaining  U.S.  researchers  among  the 
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leaders  in  long  term  fundamental  research.  The  Centers  Program  is  needed  in  our 
view  to  do  this  effectively.  As  new  advanced  computer  hardware  becomes  commer- 
cially available,  scalability  will  continue  to  be  a  challenge  for  the  foreseeable  future. 

Each  new  generation  of  computing  power  brings  with  it  vast  new  opportunities 
for  advances  in  fundamental  understanding  and  synthesis  of  complex — often  inter- 
disciplinary— research  aieas.  The  take-home  message  from  our  report  is  that  lead- 
ing edge  computing  has  and  will  continue  to  make  significant  contributions  to  the 
advance  of  science  and  engineering. 

Our  second  recommendation  is  that  NSF  needs  to  support  a  few  "Leading  Edge" 
sites.  Sites  that  have  a  balanced  set  of  high-end  hardware  capabilities  coupled  with 
appropriate  staff  and  software.  Raw  processor  speed  or  compute  power  is  not  suffi- 
cient to  achieve  the  balanced  high-performance  systems  needed  for  paradigm  shift- 
ing research. 

In  our  third  recommendation  we  focus  on  partnerships.  Partnerships  will  be  im- 
portant for  the  future  of  the  centers  program.  The  strategic  coupling  between  lead- 
ing-edge sites  and  research  centers  and  university  laboratories  is  a  significant  new 
change  that  we  are  recommending.  These  partnership  sites  will  provide  better  cou- 
pling to  computer  scientists  developing  new  tools  and  software — this  coupling  will 
be  particularly  important  when  their  research  on  mid-range  systems  reaches  the 
point  when  it  needs  to  examine  scalability  issues  that  become  manifest  only  on  the 
leading-edge  systems.  But  the  impact  of  this  research  will  extend  beyond  computer 
science.  The  partnership  sites  will  also  provide  more  cost  effective  and  diverse  plat- 
forms for  early  design  and  testing  of  applications  software  prior  to  the  stage  where 
access  to  a  leading-edge  system  is  required. 

It  has  long  been  recognized  that  some  portion  of  the  resources  at  the  NSF  Centers 
was  being  used  for  jobs  that  did  not  require  access  to  leading-edge  systems.  This 
is  particularly  true  during  the  startup,  design  and  test  stages  of  a  project — even  for 
projects  with  significant  overall  computational  requirements.  With  the  development 
of  mid-range  systems  that  are  upwardly  compatible  with  the  leading-edge  systems, 
it  should  now  be  possible  to  make  more  effective  use  of  the  leading-edge  systems 
by  reserving  them,  as  much  as  possible,  for  computations  that  require  that  unique 
level  of  capability.  The  resources  on  these  mid-range  systems  should  also  be  used 
to  obtain  preliminary  data  on  software  performance  that  would  guide  the  allocations 
committees  in  their  review  of  requests  for  access  to  leading-edge  systems. 

Recommendation  four  deals  with  the  need  for  a  new  competition  and  for  evalua- 
tion of  the  overall  program  on  a  regular  basis.  We  believe  that  a  new  competition 
is  the  best  way  to  proceed.  A  new  competition  can  be  fair  to  all  and  stimulate  the 
greatest  levels  of  creativity  in  proposal  preparation. 

Given  the  rapid  advance  of  computing  hardware  and  communications  technology, 
there  will  be  a  continuing  need  to  review  the  centers  program.  NSF  will  need  to 
carry  out  regular  annual  reviews,  as  in  the  past,  and  probably  a  full  program  review 
in  about  five  years. 

Recommendation  five  deals  with  the  support  of  research  at  the  centers.  Centers 
need  to  have  high  levels  of  expertise.  This  requires  significant  staff  involvement 
with  research,  but  not  direct  funding  of  independent  research  by  the  Centers  Pro- 
gram. 

Recommendations  six  and  seven  are  suggestions  to  NSF  management  on  leader- 
ship issues  that  the  Task  Force  believes  that  management  needs  to  focus  on,  but 
I  do  not  believe  that  they  are  central  to  these  hearings. 

Although  we  began  with  the  work  of  earlier  reports,  we  did  not  accept  their  con- 
clusions or  recommendations  uncritically.  We  carried  out  our  own  analysis,  particu- 
larly of  the  merit  review  process  for  allocating  resources  at  the  current  centers,  an 
analysis  of  the  educational  benefits  of  the  program,  and  we  also  sought  significant 
input  from  the  community,  including  several  key  NSF  Advisoryy  Committees,  on  the 
benefits  of  the  program. 

In  preparing  for  is  hearing  I  have  been  able  to  talk  with  most  of  the  members 
of  the  Task  Force  to  confirm  their  enthusiasm  for  the  manner  in  which  NSF  man- 
agement is  moving  forward  with  the  program  announcement  and  the  receipt  of  pro- 
posals. All  indicated  that  they  were  pleased  with  the  progress  that  has  been  made 
up  to  this  point. 

Thank  you  for  your  attention.  I  will  be  happy  to  answer  questions. 
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Program  Solicitation 
Partnerships  for  Advanced  Computational  Infrastructure 


Overview 

NSF  support  of  high  performance  computing  has  played  a 
major  role  m  advancing  science  and  engineering  research  and  in 
enabhng  U.  S.  world  leadership  in  computational  science  and 
engineering.  The  NSF  Supercomputer  Centers  program  has 
served  as  a  cornerstone  of  these  advances  by  expanding  the  use 
of  high-end  computing  in  new  disciplines,  facilitating  the  accep- 
tance of  computation  as  a  full  partner  in  scientific  research,  and 
facilitating  the  education  of  a  new  generation  of  computational 
scientists  and  engineers  in  support  of  that  shift. 

This  program  solicitation  for  the  Partnerships  for  Advanced 
Computational  Infrastructure  program,  builds  on  and  replaces 
the  current  NSF  Supercomputer  Centers  program  established  in 
1985,  and  focuses  on  taking  advantage  of  newly  emerging  op- 
portunities in  high  performance  computing  and  communica- 
tions. This  new  program  will  provide  flexibility,  both  to  adapt  to 
rapidly  evolving  circumstances  and  to  meet  the  need  for  high-end 
compulation,  in  order  to  enable  continued  world  leadership  in 
computational  science  and  engineering.  The  program  will  pro- 
vide access  for  researchers  to  high  performance  computing  sys- 
tems as  its  core,  with  associated  highly  trained  staff  and 
researchers  necessary  to  develop  and  optimize  their  use  The 
emergence  of  scalable  parallel  systems,  high  performance  net- 
working and  high  bandwidth,  large  capacity  mass  storage  sys- 
tems creates  the  opportunity  for  a  national  infrastructure 
consisting  of  a  number  of  geographically  distributed  sues 
strongly  coupled  to  high-end  computational  resources  and  to 
each  other  via  high-speed  communication  networks. 

NSF  envisions  an  Advanced  Computational  Infrastructure 
consisting  of  one  or  more  leading-edge  sites  together  with  coop- 
erating partners.  Leading-edge  sites  are  expected  to  maintain 
high-end  hardware  systems  that  are  one  to  two  orders  of  magni- 
tude more  capable  than  those  typically  available  at  a  major 
research  university.  These  systems  should  be  balanced  in  terms 
of  processor  speed,  memory,  and  storage  systems,  and  should  be 
accompanied  by  appropriate  staff,  software  and  high  speed  com- 
munications capability.  The  partners  will,  in  the  aggregate, 
complete  the  overall  infrastructure  by,  among  other  things, 
(a)  facilitating  research  and  experimentation  with  new  hardware 
and  software,  including  appropriate  support  technologies  such  as 
visualization  and  mass  storage,  (b)  providing  scalable  resources 
for  applications  and  applications  development  that  can  be  best 
done  on  mid-level  systems,  (c)  providing  access  to  unique  ex- 
perimental systems  and  facilities,  and  (d)  promoting  education 
and  training. 

Background 

Several  reports  analyzing  the  current  Supercomputer  Centers 
Program  and  making  recommendations  for  future  computational 
infrastructure  have  appeared  recently.  Most  directly  relevant  to 
this  solicitation  is  the  Report  of  the  Task  Force  on  the  Future  of 


the  NSF  Supercomputer  Centers  Program,  chaired  by  Dr  Ed- 
ward Hayes,  September,  1995,  which  discusses  the  history,  cur- 
rent context,  and  rationale  for  NSF  support  of  high  performance 
computational  infrastructure  for  the  science  and  engineering 
research  communities. 

Two  other  reports.  From  Desktop  to  Teraflop:  Exploiting  the 
U.S.  Lead  in  High  Performance  Computing,  chaired  by  Dr. 
Lewis  Branscomb,  October,  1993,  and  Evolving  the  High  Per- 
formance Computing  and  Communications  Initiative  to  Support 
the  Nation's  Information  Infrastructure,  chaired  by  Drs. 
Frederick  Brooks  and  Ivan  Sutherland,  National  Research  Coun- 
cil, February,  1995  provide  additional  background  and  discuss 
the  overall  impact  of  these  programs  in  the  context  of  the  larger, 
federally  supported  High  Performance  Computing  and  Commu- 
nications program. 

These  reports  are  available  on  the  World  Wide  Web  via  URL 
http://www.cise.nsf.gov/. 

Hard  copies  of  the  first  two  reports  may  be  obtained  by 
sending  e-mail  to  bmelvin@nsf  gov.  The  NRC  report  is  available 
from  the  National  Research  Council,  2101  Constitution  Ave. 
N.W.,  Washington.  DC  20418. 

Mission 

As  part  of  its  strategic  plan.  NSF  in  a  Changing  World,  a  key 
NSF  goal  IS  to  enable  the  United  States  to  uphold  a  position  of 
world  leadership  in  science  and  engineering  research  and  educa- 
tion. In  order  to  maintain  world  leadership  in  computational 
science  and  engineering,  NSF  intends  to  create  an  advanced 
national  computational  infrastructure  whose  overall  mission  is 
to: 

•  provide,  facilitate,  and  enhance  access  to  high  performance 
computational  infrastructure  for  the  U.  S.  academic,  scien- 
tific, and  engineenng  communities  by  partnering  with 
universities,  states,  and  the  private  sector; 

•  promote  vigorous  early  use  of  experimental  and  emerging 
high  performance  computational  and  associated  commu- 
nications technologies  that  offer  high  potential  for  advanc- 
ing science  and  engineering; 

•  enable  the  effective  use  of  such  infrastructure  and  tech- 
nologies through  education,  training,  consulting,  and  re- 
lated support  services,  including  appropriate  software 
development,  experimentation,  and  support; 

•  foster  interdisciplinary  research  in  science  and  engineer- 
mg; 

•  facilitate  the  development  of  the  intellectual  capital  re- 
quired to  maintain  world  leadership  in  computational  sci- 
ence and  engineering;  and 
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•  broaden  the  base  for  the  nation's  advanced  computational 
and  communications  mfraslructure. 

Purpose  of  this  Solicitation 

This  solicitation  calls  for  Innovative  proposals  to  provide  a 
national  computational  infrastructure  that  will  address  the  mis- 
sion stated  above.  Guidance  for  participating  in  this  solicitation 
IS  provided  below. 

Since  the  Partnerships  for  Advanced  Computational  Infra- 
structure Program  will  primarily  support  academic  research,  it  is 
expected  that  proposals  will  come  from,  and  the  partnership's 
cooperative  agreement  will  be  with,  a  U.  S.  academic  institution. 
However,  a  successful  proposal  must  involve  multiple  partners 
who  can  be.  but  are  not  limited  to: 

•  universities,  including  research  groups  within  universities; 

•  NSF-funded  Centers  and  facilities,  such  as.  Science  and 
Technology  Centers.  Supercomputer  Centers,  Engineering 
Research  Centers,  and  Industry-University  Cooperative 
Research  Centers; 

•  research  and  educational  consortia,  organizations,  and 
groups; 

•  regional  and  stale-supported  high-performance  computing 
centers; 

•  private  sector  organizations;  and 

•  national  laboratories. 

A  full  description  of  organizations  and  individuals  who  are 
eligible  to  submit  proposals  to  NSF  and  the  conditions  under 
which  they  may  compete,  are  given  in  the  NSF  Grant  Proposal 
Guide  (GPG)  (NSF  95-27).  Single  copies  of  this  brochure  are 
available  at  no  cost  from  the  NSF  Forms  and  Publications  Unit, 
(703)  306- 1 1 30,  or  via  e-mail:  pubs(a'nsf  gov.  Brochures  are  also 
available  electronically  through  NSF's  Science  and  Technology 
Information  System  (STIS)  and  on  the  World  Wide  Web  at  URL 
htlp://www.nsf.gov/. 

Review  Process  and  Criteria 

The  review  process  will  involve  preproposals  and  full  propos- 
als, both  of  which  will  be  reviewed  by  external  panels  using 
criteria  discussed  below.  Panelists  will  be  chosen  from  a  variety 
of  sectors  including  academia,  the  private  sector,  national  labo- 
ratories, and  other  government  agencies.  Site  visits  are  antici- 
pated for  finalists  in  the  competition  in  order  to  clarify  issues 
raised  during  the  reviews,  and  to  explore  additional  matters  as 
needed.  A  summary  panel,  consisting  of  the  chairs  of  the  site  visit 
teams  and  the  chair  of  the  final  proposal  review  panel,  will  meet 
after  the  site  visits  to  formulate  final  recommendations  for  a 
balanced  program. 

As  with  all  proposals  to  NSF,  these  proposals  will  be  evaluated 
using  the  four  standard  review  criteria:  (I)  the  quality  of  the 
proposed  scientific  effort.  (2)  competence  of  the  investigators. 
(3)  relevance  of  the  research,  and  (4)  impact  on  the  infrastructure 
of  science  and  engineering.  However,  for  this  particular  solicita- 


tion, impact  on  infrastructure  and  competence  of  investigators 
will  be  especially  important.  Therefore  preproposals  and  final 
proposals  for  this  program  will  be  subject  to  four,  more  specific, 
criteria  dealing  with  the  effectiveness  of  the  overall  partnership 
in  addressing  the  program's  mission,  the  quality  of  the  individual 
partners,  management,  and  financial  leverage.  While  these  more 
specific  criteria  are  listed  in  priority  order  (with  the  first  being 
the  most  important),  all  will  be  considered  by  the  panels  in 
arriving  at  recommendations.  Finally,  the  summary  panel  will  be 
asked  to  advise  NSF  on  what  each  proposed  partnership  can 
contribute  to  the  technical  diversity  and  balance  of  the  program 
as  a  whole.  In  assessing  the  degree  to  which  each  preproposal 
and  proposal  satisfies  a  particular  criterion,  the  reviewers  will  be 
asked  to  consider  each  of  the  following  points. 

1 )  The  degree  to  which  the  proposed  partnership  addresses  the 
overall  mission  statement  and  demonstrates  that  it  will  be  able 
to: 

•  provide  the  physical  and  human  infrastructure  needed  to 
fulfill  a  national  leadership  role  with  an  intellectual  envi- 
ronment that  can  foster  world  leadership  in  computational 
science  and  engineering  research  across  all  NSF  disci- 
plines; 

•  maintain  balanced  computational  resources,  including  at 
least  one  site  with  capabilities  one  to  two  orders  of  magni- 
tude greater  than  that  typically  available  at  a  major  re- 
search university; 

•  enable  interdisciplinary  partnerships  among  the  academic 
computer  science,  mathematics,  and  computational  sci- 
ence research  communities; 

•  support  and  develop  software  enabling  computational  sci- 
ence and  engineering; 

•  effect  outreach  and  technology  transfer  to  a  heterogeneous 
user  base; 

•  support  cooperative  relationships  with  hardware  and  soft- 
ware vendors; 

•  foster  the  integration  of  research,  education,  and  training 
in  computational  science  and  engineering; 

•  promote  the  advancement  of  kindergarten  through  under- 
graduate science  and  engineering  education  generally;  and 

•  provide  world-class  leadership  through  a  diverse  and  syn- 
ergistic set  of  partners. 

2)  The  degree  to  which  the  proposed  individual  partners  demon- 
strate expertise  in  at  least  one  of  the  following: 

•  providing  access  to.  and  support  for.  high  performance 
computing,  including  appropriate  supporting  technolo- 
gies; 

•  conducting  world-class  research  in  computational  science 
and  engineering,  computer  science  and  engineering,  or  an 
appropriate  related  field; 
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•  promoting  vigorous  early  use  of  experimental  and  emerg- 
ing scalable  computing,  communication,  or  mass  storage 
technologies; 

•  providing  training,  education,  and  outreach  to  the  research 
and  educational  communities  and  to  the  private  sector:  and 

•  supporting  cooperative  efforts  across  multiple  intellectual 
and/or  institutional  sectors. 

3)  The  quality  of  the  proposed  management  of  the  partnership 
including: 

•  clear  plans  for  the  close  coordination  and  management  of 
the  partnership,  including  the  computational,  communica- 
tions, and  intellectual  resources  of  the  partners;  and 

•  plans  for  measuring  the  partnership's  success  in  meeting 
its  goals. 

4)  The  degree  of  financial  leverage  in  the  proposed  program, 
including  cost  sharing.  This  can  include  personnel  provided 
by  institutions  and  vendors,  state  and  local  support,  facilities, 
vendor  discounts  beyond  normal  educational  discounts,  soft- 
ware, etc. 

In  addition,  the  final  selection  by  NSF  will  consider  what  each 
partnership  can  contribute  to  the  technical  diversity  and  balance 
of  the  program  as  a  whole. 

Funding  Levels  and  Duration  of  Awards 

NSF  funding  for  the  current  NSF  Supercomputer  Centers 
Program  totals  about  $65  million  per  year.  Funding  levels  for  the 
new  program  will  depend  on  expected  overall  funding  levels  for 
NSF  at  the  time  of  the  award(s).  NSF  funding  will  enable  active 
participation  in  the  partnership  and  create  a  broad  base  support- 
ing the  national  computational  infrastructure  for  research  in 
science  and  engineering.  The  number  and  size  of  awards  will  be 
based  on  the  quality  and  potential  impact  of  the  proposed  part- 
nerships, and  on  the  availability  of  funds.  Each  partnership  will 
be  funded  through  a  single  cooperative  agreement  specifying  the 
level  of  support  for  each  partner,  as  proposed  by  the  partnership, 
and  negotiated  with  NSF  before  the  award.  The  cooperative 
agreement  for  the  partnerships  is  expected  to  cover  a  five-year 
period  beginning  in  fiscal  year  1998. 

Phase  in  of  a  new  partnership  is  expected  to  occur  over  a  one 
year  period.  Partnership  plans  and  proposed  budgets  must  be 
prepared  with  enough  flexibility  to  allow  the  phasing  in  of 
partnership  resources  during  the  first  year,  as  negotiated  with 
NSF.  To  provide  flexibility  in  meeting  special  needs,  funding  can 
be  provided  for  partnerships  having  a  membership  that  may 
change  over  the  duration  of  the  cooperative  agreement;  thus, 
proposals  involving  partners  for  a  shorter  period  are  encouraged, 
where  appropriate.  Funding  will  be  provided  in  yearly  incre- 
ments subject  to  successful  annual  review  of  program  plans, 
performance,  and  the  availability  of  funds. 

To  provide  a  partnership  with  the  flexibility  to  respond  to  an 
unusual  opportunity,  completion  of  a  partner's  task,  changes  or 
relocation  of  a  critical  person,  etc.,  the  partnership  may  propose 
(at  any  time,  subject  to  review),  a  change  in  the  number  or 


composition  of  its  partner  sites.  Normally  such  proposals  will  be 
pan  of  the  annual  program  plan  review  process  and  would  be 
considered  along  with  other  substantive  issues. 

The  partnerships  will  be  reviewed  annually  by  an  external 
partnership  review  panel,  with  site  visits  in  the  second  year.  In 
the  fourth  year  of  the  program,  NSF  will  conduct  a  full  inde- 
pendent review  of  the  overall  program.  Subject  to  the  needs  of 
the  scientific  and  engineering  community  as  determined  by  this 
review;  the  acceptable  annual  progress  and  performance  reviews 
of  the  partnerships;  and  the  availability  of  funds,  NSF  may  invite 
renewal  proposals,  with  the  intent  of  extending  successful  part- 
nerships for  an  additional  five-year  period. 

At  the  end  of  ten  years,  this  program  will  be  "sunset" .  During 
the  eighth  year  there  will  be  a  full  and  independent  review  to 
determine  the  anticipated  future  needs  of  the  academic  science 
and  engineering  community 

Program  IVIanagement 

Each  partnership  will  offer  a  variety  of  resources.  In  addition 
to  computational  resources,  there  might  be  visualization  re- 
sources, such  as  access  to  a  virtual  reality  environment,  mass 
storage  resources  located  at  a  single  site,  discipline-specific 
knowledge  repositones,  software  development  resources,  code 
porting  and  optimization  resources,  discipline-specific  subrou- 
tine and  program  libraries,  performance  instrumentation  tools 
and  libraries,  generic  subroutine  libraries  such  as  linear  algebra 
codes,  and  educational  and  training  resources. 

The  individual  partnerships  are  being  asked  to  propose  man- 
agement plans,  allocation  mechanisms,  and  means  for  obtaining 
external  advice.  However,  NSF  will  require  that  the  partnerships 
develop  a  single  mechanism  for  allocation  of  the  use  of  the  bulk 
of  the  program's  resources,  independent  of  where  they  are  lo- 
cated 

Budget 

Funds  may  be  requested  to  support  scientific  and  technical 
staff  essential  for  systems  programming,  software  development, 
research  and  user  services,  coordination  and  conduct  of  educa- 
tion, training  and  outreach  activities,  and  participant  support; 
training  materials  and  software;  networking  equipment  and  op- 
erating costs;  and  computer  systems,  indirect  costs  and  other 
appropriate  costs  associated  with  the  overall  project.  Proposers 
should  Itemize  costs  on  the  NSF  Budget  Form  1030  and  include 
additional  supporting  documentation  Proposed  cost  sharing 
should  be  recorded  on  Line  M  of  the  Budget  Form  1030. 

Cost  Sharing 

Cost  sharing  may  be  in  cash  or  in-kind  from  any  private  or 
non-federal  source.  The  estimated  value  of  any  in-kind  contribu- 
tions should  be  included  and  an  explanation  of  the  source,  nature, 
amount,  and  availability  of  any  proposed  cost  sharing  should  also 
be  provided.  Cost  sharing  must  occur  during  the  award  period 
and  must  be  consistent  with  OMB  Circular  A- 1 10,  Section  .23. 
Cost  sharing  specified  in  the  proposal  will  be  referenced  and 
included  as  a  condition  of  the  award. 
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Preproposal  Guidelines 

In  order  to  be  eligible  to  submit  a  formal  proposal,  the  institu- 
tion must  have  submitted,  by  April  15,  1996,  a  preproposal  that 
was  reviewed,  or  been  named  as  a  partner  on  a  preproposal  that 
was  reviewed.  Preproposals  will  be  reviewed  by  external  panels 
using  the  criteria  listed  above.  The  intent  of  the  preproposal 
round  is  to  give  the  community  an  opportunity  to  put  forth 
diverse  ideas,  provide  information  that  will  assist  in  developing 
a  stronger  proposal,  and  point  out  preproposals  that  are  not 
expected  to  be  competitive.  Review  comments  from  the  prepro- 
posal round  will  be  supplied  about  the  middle  of  May  1996  to 
the  proposers  in  order  to  provide  feedback  for  the  final  proposal 
preparation.  Multiple  preproposals  from  the  same  proposing 
organization  involving  different  partners,  funding  levels,  and 
objectives  are  allowed  Neither  institutional  approvals  nor  signa- 
tures are  required  at  this  stage;  however,  proposals  should  follow 
any  applicable  institutional  guidelines  for  preproposals. 

Preproposals  should  be  mailed  to; 

Partnerships  in  Advanced  Computational  Infrastructure 
NSF,  Room  1 1 22 
4201  Wilson  Boulevard 
Arlington,  VA  22230 

Preproposals  must  be  received  by  NSF  no  later  than  April  15, 
1996;  or  postmarked  no  later  than  five  (5)  days  prior  to  the 
deadline  date;  or  sent  via  commercial  overnight  mail  no  later  than 
two  (2)  days  prior  to  the  deadline  date,  to  be  considered  for 
review. 

Preproposals  submitted  in  response  to  this  solicitation  must  be 
prepared  and  submitted  in  accordance  with  the  guidelines  pro- 
vided in  the  NSF  Gram  Proposal  Guide.  (GPG )  (NSF  95-27);  in 
particular,  page  formatting  requirements  given  in  Chapter  2, 
Section  C,  will  be  strictly  enforced  and  preproposals  not  com- 
plying will  be  returned  without  review.  Preproposals  should 
allow  reviewers  to  address  each  of  the  evaluation  criteria  listed 
above. 

Preproposals  (15  copies)  must  include: 

•  A  cover  page  listing  the  Pi's  e-mail  address  and  FAX 
number,  and  all  participating  sites  and  organizations; 

•  An  executive  summary  of  2-3  pages; 

•  A  main  body  of  10  single-spaced  pages,  to  include: 

-  goals  and  objectives; 

-  a  description  of  the  proposed  partnership  activities; 

-  an  overview  of  the  partnership,  its  participants,  and 
plans  for  closely  coordinating  the  participants; 

-  a  staffing  description  and  summary  of  the  key  person- 
nel at  each  site; 

-  a  list  of  the  expected  accomplishments  of  the  partner- 
ship over  the  five-year  period  of  the  award; 

-  a  proposed  management  plan  for  the  partnership,  in- 
cluding mechanisms  for  making  policy  and  planning 
decisions,  and  obtaining  advice  from  the  community; 


-  a  proposed  method  for  allocating  and  closely  coordi- 
nating the  computational  and  intellectual  resources  of 
the  partnership;  and 

-  a  plan  for  evaluating  the  success  of  the  partnership. 

•  Adescription  of  proposed  hardware,  networking,  and  soft- 
ware, including  planned  upgrades; 

•  Key  planned  software  and  algorithm  developments,  if 
applicable; 

•  Cost  sharing  projections,  both  cash  and  in  kind; 

•  A  non-binding  draft  budget  for  each  year  (the  proposed 
funding  level  for  each  partner  should  be  specified); 

•  An  appendix  containing  one  to  two  page  vita  including  the 
most  relevant  publications  for  the  key  personnel;  and 

•  No  additional  appendices  are  allowed. 

The  preproposals  will  be  reviewed  by  external  panels.  It  is 
anticipated  that  preproposal  review  will  be  completed  and  feed- 
back provided  to  the  proposers  by  mid-May  1996. 

Proposal  Guidelines 

To  be  eligible  to  submit  a  formal  proposal,  the  institution  must 
have  submitted  a  preproposal  that  was  reviewed,  or  have  been  a 
partner  on  a  preproposal  that  was  reviewed. 

Proposals  should  be  mailed  to: 

Partnerships  in  Advanced  Computational  Infrastructure 
NSF  Room  1122 
4201  Wilson  Boulevard 
Arlington,  VA  22230 

Proposals  must  be;  received  by  NSF  no  later  than  September  I , 
1996;  or  postmarked  no  later  than  five  (5)  days  prior  to  the 
deadline  date;  or  sent  via  commercial  overnight  mail  no  later  than 
two  (2)  days  prior  to  the  deadline  date,  to  be  considered  for 
review. 

Proposals  submitted  in  response  to  this  solicitation  must  be 
prepared  and  submitted  in  accordance  with  the  guidelines  pro- 
vided in  the  NSF  Grant  Proposal  Guide  (GPG)  (NSF  95-27);  in 
particular,  page  formatting  requirements  given  on  page  3,  Sec- 
tion C,  will  be  strictly  enforced  and  proposals  not  complying  will 
be  returned  without  review.  Proposals  should  allow  reviewers  to 
address  each  evaluation  criterion  listed  above. 

Proposals  (15  copies)  should  include  the  following: 

•  A  cover  page  listing  the  Pi's  e-mail  address  and  FAX 
number,  and  all  participating  sites  and  organizations; 

•  An  executive  summary  of  2-3  pages; 

•  A  main  body  of  40  single-spaced  pages,  to  include: 

-  goals  and  objectives; 

-  a  description  of  the  proposed  partnership  activities; 

-  an  overview  of  the  partnership,  its  participants,  and 
plans  for  closely  coordinating  the  participants; 
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-  a  staffing  description  and  summary  of  the  key  person- 
nel at  each  site; 

■  a  hst  of  the  expected  accomplishments  of  the  partner- 
ship over  the  five-year  period  of  the  award; 

-  a  proposed  management  plan  for  the  partnership  in- 
cluding mechanisms  for  making  policy  and  planning 
decisions,  and  obtaining  advice  from  the  community; 
a  proposed  method  for  allocating  and  closely  coordi- 
nating the  computational,  and  intellectual  resources  of 
the  partnership;  and 

-  a  plan  for  evaluating  the  success  of  the  partnership. 

•  A  description  of  proposed  hardware,  networking,  and  soft- 
ware, including  planned  upgrades; 

•  Key  planned  software  and  algorithm  developments,  if 
applicable; 

•  Cost  sharing  proposed,  both  cash  and  in  kind; 

•  Abudget  for  each  year(the  proposed  funding  level  for  each 
partner  should  be  specified); 

•  An  appendix  containing  one  to  two  page  vita  including  the 
most  relevant  publications  for  the  key  personnel; 

•  An  appendix  containing  letters  that  commit  actual  re- 
sources such  as  funds,  hardware,  people,  space,  etc.  from 
the  participating  institutions,  vendors,  states,  etc.  and 

•  No  additional  appendices  are  allowed. 

Proposals  will  be  reviewed  by  external  panels.  While  there  will 
be  some  overlap  between  the  panels  reviewing  the  preproposals 
and  final  proposals,  the  panelists  will  be  instructed  that  any 
recommendations  based  on  the  preproposal  review  were  not 
intended  to  be  binding  on  the  proposer  Recommendations  and 
reviews  from  the  preproposal  round  will  not  be  available  to  the 
final  proposal  panel,  and  should  not  be  considered  in  reviewing 
the  final  proposals.  Site  visits  are  anticipated  for  finalists  in  the 
competition  in  order  to  clarify  issues  raised  during  the  reviews, 
and  to  explore  additional  matters  as  needed. 

Panel  reviews  are  expected  to  be  completed  by  October,  1996, 
and  site  visits  will  occur  before  the  end  of  the  1 996  calendar  year. 
Awards  should  be  announced  in  the  Spring  of  1997. 

Inquiries  related  to  the  program  should  be  sent  via  e-mail  to 
PACI(s)nsfgov 

or  they  may  be  sent  by  US  mail  to: 

Partnerships  in  Advanced  Computational  Infrastructure 
NSF,  Room  1122 
4201  Wilson  Boulevard 
Arlington,  VA  22230 


GENERAL  INFORMATION 

The  Foundation  provides  awards  for  research  in  the  sciences 
and  engineering.  The  awardee  is  wholly  responsible  for  the 
conduct  of  such  research  and  preparation  of  the  results  for 
publication.  The  Foundation,  therefore,  does  not  assume  respon- 
sibility for  the  research  findings  or  their  interpretation. 

The  Foundation  welcomes  proposals  from  all  qualified  scien- 
tists and  engineers  and  strongly  encourages  women,  minorities, 
and  persons  with  disabilities  to  compete  fully  in  any  of  the 
research  related  programs  described  here. 

In  accordance  with  federal  statues,  regulations,  and  NSF  poli- 
cies, no  person  on  grounds  of  race,  color,  age,  sex,  national 
origin,  or  disability  shall  be  excluded  from  participation  in.  be 
denied  the  benefits  of,  or  be  subject  to  discrimination  under  any 
program  or  activity  receiving  financial  assistance  from  the  Na- 
tional Science  Foundation. 

Facilitation  Awards  for  Scientists  and  Engineers  with  Dis- 
abilities (FASEDl  provide  funding  for  special  assistance  or 
equipment  to  enable  persons  with  disabilities  (investigators  and 
other  staff,  including  student  research  assistants)  to  work  on  NSF 
projects.  See  the  program  announcement  ( NSF  9 1  -54 )  or  contact 
the  program  coordinator  at  306-1636. 

Privacy  Act  and  Public  Burden.  The  information  requested 
on  proposal  forms  is  solicited  under  the  authority  of  the  National 
Science  Foundation  Act  of  1950,  as  amended.  It  will  be  used  in 
connection  with  the  selection  of  qualified  proposals  and  may  be 
disclosed  to  qualified  reviewers  and  staff  assistants  as  part  of  the 
review  process;  to  applicant  institutions/grantees  to  provide  or 
obtain  data  regarding  the  application  review  process,  award 
decisions,  or  the  administration  of  awards;  to  government  con- 
tractors experts,  volunteers,  and  researchers  as  necessary  to 
complete  assigned  work;  and  to  other  government  agencies  in 
order  to  coordinate  programs.  See  Systems  of  Records.  NSF  50, 
Principal  Investigators/Proposal  File  and  Associated  Re- 
cords, and  NSF-51.  60  Federal  Regislei  4449  (January  23, 
1995)  Reviewer/Proposal  File  and  Associated  Records,  59 
Federal  Register  8031  (February  17,  1994).  Submission  of  the 
information  is  voluntary.  Failure  to  provide  full  and  complete 
information,  however,  may  reduce  the  possibility  of  your  receiv- 
ing an  award. 

Public  reporting  burden  for  this  collection  of  information  is 
estimated  to  average  120  hours  per  response,  including  the  time 
for  reviewing  instructions.  Send  comments  regarding  this  burden 
estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Herman  G. 
Fleming.  Reports  Clearance  Officer.  Contracts,  Policy,  and 
Oversight,  National  Science  Foundation,  4201  Wilson  Boule- 
vard, Arlington,  VA  22230. 

The  National  Science  Foundation  has  TDD  and  FIRS  capabil- 
ity, which  enables  individuals  with  hearing  impairment  to  com- 
municate with  the  Foundation  about  NSF  programs, 
employment,  or  general  information.  To  access  TDD  dial  (703) 
306-0090;  for  FIRS,  1-800-877-8339. 
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6ETTIN6  NSF  INFORMATION  AND  PUBLICATIONS 

The  National  Science  Foundation  (NSF)  has  several  ways  for  the  public  to  receive  information 
and  publications.  Electronic  or  printed  copies  of  the  NSF  telephone  directory,  abstracts  of 
awards  made  since  1989,  and  many  NSF  publications  are  available  as  described  below.  To 
access  information  electronically,  there  is  no  cost  to  you  except  for  possible  phone  and  Internet 
access  charges.  Choose  the  method  of  access  that  matches  your  computer  and  network  tools.  For 
general  information  about  Internet  access  and  Internet  tools,  please  contact  your  local  computer 
support  organization. 


WORLD  WIDE  web: 
NSF  HOME  PACE 

The  World  Wide  Web  (WWW)  syslem 
makes  it  possible  to  view  text  material 
as  well  as  graphics,  video,  and  sound. 
You  will  need  special  software  (a  "web 
browser")  to  access  the  NSF  Home 
Page.  The  URL  (Uniform  Resource 
Locator)  is  http://www.n5f.g0v/. 

INTERNET  «OPHER 

The  Internet  Gopher  provides  access  to 
information  on  NSF's  Science  and 
Technology  Information  System 
(STIS)  through  a  series  of  menus.  To 
access  the  Gopher,  you  need  Gopher 
client  software;  the  NSF  Gopher  server 
Is  on  port  70  ofstis.nsf.gov. 

ANONYMOUS  FTP  (FILE 
TRANSFER  PROCRAM) 

Internet  users  who  are  familiar  with 

FTP  can  easily  transfer  NSF 

documents  to  their  local  system  for 

browsing  and  printing.  The  best  way 

to  access  NSF  information  is  to  first 

look  at  the  index  (file  name: 

index.txt).  From  the  index,  you  can 

select  the  files  you  need.  FTP 

instructions  are: 

■    FTPtostis.nsf.gov. 

m    Enter  anonymous  for  the  user  name. 

and  your  e-mail  address  for  the 

password. 
•    Retrieve  the  appropriate  file  (i.e., 

filename.ext) 

E-MAIL  (ELECTRONIC-MAIL) 

To  get  documents  via  e-mail,  send  your 
request  to  the  Internet  address 
slisserve@nsf.gov    The  best  way  to 
find  NSF  information  is  to  request  the 
index.  Your  e-mail  message  should 
read:  get  index.txt.  An  index  with  file 
names  will  be  sent  to  you.  However  if 
you  know  the  file  name  of  the 
document  you  want,  your  e-mail 
message  should  read: 
get  <filenanie.ext> 


E-MAIL  MAILINC  LISTS 

NSF  maintains  several  mailing  lists  to 
keep  you  automatically  informed  of 
new  electronic  publications.  To  get 
descriptions  of  the  mail  lists  and 
instructions  for  subscribing,  send  your 
request  to:  slisserve@nsf.gov.  Your 
message  should  read:  get  stisdirm.txt. 

ON-LINE  STIS 

NSF's  Science  and  Technology 
Information  System  (STIS)  is  an 
electronic  publications  dissemination 
system  available  via  the  Internet  (telnet 
to  slis.nsf.gov),  you  will  need  a  VTIOO 
emulator.  The  system  features  a  full- 
text  search  and  retrieval  software 
(TOPIC)  to  help  you  locate  the 
documents.  Login  as  public  and  follow 
the  instructions  on  the  screen. 

To  get  an  electronic  copy  of  the  "STIS 
USERS  GUIDE."  NSF  94-10.  send  an 
e-mail  request  to:  stisserve@nsf.gov. 
Your  message  should  read: 
get  NSF9410.txt.  For  a  printed  copy  of 
the  "STIS  USERS  GUIDE,"  see 
instructions  "How  To  Request  Printed 
NSF  Publications." 


NON-INTERNET  ACCESS 
VIA  MODEM 

If  you  do  not  have  an  Internet 
connection,  you  can  use  remote  login 
to  access  NSF  publications  on  NSF's 
on-line  system,  STIS  You  need  a 
VTIOO  terminal  emulator  on  your 
computer  and  a  modem. 

•  Dial  703-306-0212, 

•  choose  1 200,  2400,  or  9600  baud, 

•  use  settings  7-E- 1 ,  and 

•  login  as  public  and  follow  the  on- 
screen instructions. 


NSF  95-64  (Replaces  NSF  94-4) 


HOW  TO  REQUEST  PRINTED 
NSF  PUBLICATIONS 

You  may  request  printed  publications 
in  the  following  ways: 

•  send  e-mail  request  to: 
pubs@nsf.gov 

m    fax  request  to:  703-644-4278 

■  for  phone  request,  call:  703-306- 
1130  or  Telephonic  Device  for  the 
Deaf  (TDD  703-306-0090) 

■  send  written  request  to: 

NSF  Forms  and  Publications  Unit 
4201  Wilson  Boulevard 
Room  P- 15 
Arlington,  VA  22230 

When  making  a  request,  please  include 
the  following  information: 

•  NSF  publication  number; 

•  number  of  copies;  and 

•  your  complete  mailing  address. 

QUESTIONS  ABOUT  NSF 
PUBLICATIONS.  PROCRAMS/ 
ETC 

Contact  the  NSF  Information  Center  if 
you  have  questions  about  publications, 
including  publication  availability, 
titles,  and  numbers.  The  NSF 
Information  Center  maintains  a  supply 
of  many  NSF  publications  for  public 
use.  You  may: 

•  visit  the  NSF  Information  Center, 
located  on  the  second  fioor  at  4201 
Wilson  Blvd.,  Arlington,  Virginia.; 
or 

•  call  the  NSF  Information  Center  at 
703-306-1234;  or  703-306-0090  for 
TDD;  or 

•  send  e-mail  message  to 
info@nsf.gov 

QUESTIONS  ABOUT  THE 
ELECTRONIC  SYSTEM 

Send  specific,  system-related  questions 
about  NSF  electronic  publication 
services  that  are  not  answered  in  this 
fiyer,  to  webmaster@nsf.gov  or  call 
703-306-0214  (voice  mail). 
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The  NSF  supported  Supercomputer  Cen- 
ters have  played  a  major  role  in  advanc- 
ing science  and  engineering  research. 
They  have  enabled  collaboration  among 
academic,  industrial  and  government  re- 
searchers on  the  solution  of  problems  re- 
quiring demanding  computational  and 
visualization  tools.  In  the  10  years  of 
their  existence,  the  Centers  have  fostered 
fundamental  advances  in  our  under- 
standing of  science  and  engineering,  ex- 
panded the  use  of  high-end  computing  in 
new  disciplines,  facilitated  the  major 
paradigm  shift  of  the  acceptance  of  com- 
putational science  as  a  full  partner  in  the 
.scientific  method,  and  facilitated  the  edu- 
cation of  a  new  generation  of  computa- 
tional scientists  and  engineers  in  support 
of  that  shift.  These  statements  are  docu- 
mented in  the  body  of  the  report  as  well 
as  in  its  appendices. 

Having  accomplished  so  much  in  the  la.st 
decade,  it  is  natural  to  ask  what  the  future 
role  of  the  NSF  should  be  in  high-end 
computing.  In  October  1994.  the  Na- 
tional Science  Board  approved  two-year 
contmuation  funding  for  the  Supercom- 
puter Centers.  This  provided  time  for  the 
director  of  NSF  to  appomt  this  Task 
Force  to  analyze  the  alternatives.  The 
possibilities  considered  include  continua- 
tion, restructuring,  or  phasing-out  of  the 
current  program,  as  well  as  creation  of 
alternative  models. 

The  Task  Force  believes  that  the  future 
for  computational  science  and  engineer- 
ing can  be  as  bright  or  even  brighter  than 
in  the  past  decade.  If  we  seize  the  op- 
portunity, over  the  next  decade  we  can 
make  major  progress  on  multiple  fronts. 

There  will  be 

•  opportunities  for  exciting  applica- 
tions of  our  nation's  exponentially 
increasing  computational  capacity, 
for  example: 


-  more  complete  models,  and  hence 
deeper  understanding  of  physical 
systems  by  moving  to  three  and 
higher  dimensions; 

-  progress  in  computational  tools  to 
aid  drug  and  protein  design; 

-  computational  predictions  of  sci- 
entifically and  commercially  sig- 
nificant materials; 

-  multidiscipiinary  models  of 
physical  systems  (e.g.,  combining 
fluid  dynamics  and  electromag- 
netic models  of  the  heart); 

-  increased  interconnectivity  of  su- 
percomputers and  high  impact  in- 
.strumentation;  and 

-  models  of  anatomical  and 
physiological  proces.ses  leading  to 
new  insights  of  benefit  to  human 
health. 

•  more  quantitative  computational 
results  in  unanticipated  areas. 

•  more  explosive  growth  of  com- 
munications as  a  component  of  the 
computational  .science  and  engi- 
neering paradigm;  and.  impor- 
tantly. 

•  continued  progress  in  the  tools  and 
methods  for  developing  code  that 
is  both  portable  and  yet  takes  ad- 
vantage of  unique  parallel  archi- 
tectures. 

These  advances  will  not  automatically  be- 
come available  to  American  researchers. 
To  position  the  U.S.  academic  commu- 
nity to  participate  in  the  exciting  research 
possibilities  enabled  by  these  develop- 
ments, the  Task  Force  has  the  following 
recommendations  leading  to  a  restruc- 
tured Centers  program. 


September  15,  1995 
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In  order  to  maintain  world  leadership 
in  computational  science  and  engi- 
neering, NSF  should  continue  to 
maintain  a  strong,  viable  Advanced  Sci- 
entific Computing  Centers  program, 
whose  mission  is: 

•  providing  access  to  high-end 
computing  infrastructure  for  the 
academic  scientific  and  engi- 
neering community; 

•  partnering  with  universities, 
states,  and  industry  to  facilitate 
and  enhance  that  access: 

•  supporting  the  effective  use  of 
such  infrastructure  through 
training,  consulting,  and  related 
support  services: 

•  being  a  vigorous  early  user  of 
experimental  and  emerging  high 
performance  technologies  that 
offer  high  potential  for  advanc- 
ing computational  science  and 
engineering; 

•  facilitating  the  development  of  the 
intellectual  capital  required  to 
maintain  world  leadership. 

NSF  should  assure  that  the  Centers 
program  provides  national  "Leading- 
edge  sites"  that  have  a  balanced  set  of 
high-end  hardware  capabilities,  coupled 
with  appropriate  staff  and  software, 
needed  for  continued  rapid  advance- 
ment in  computational  science  and  en- 
gineering. 

NSF,  through  its  Centers  program, 
should  assure  that  each  leading-edge 
site  is  partnered  with  experimental  fa- 
cilities at  universities,  NSF  research 
centers,  and/or  national  and  regional 
high  performance  computing  centers. 
Appropriate  funding  should  be  provided 
for  the  partnership  sites. 

NSF  should  announce  a  new  com- 
petition of  the  High  Performance  Com- 


puting Centers  program  that  would 
permit  funding  of  selected  sites  for  a 
period  of  five  years.  If  regular  reviews 
of  the  Program  and  the  selected  sites 
are  favorable,  it  should  be  possible  to 
extend  initial  awards  for  an  additional 
five  years  without  a  full  competition. 

The  Centers  program  should  continue 
to  support  need-based  research  in  sup- 
port of  the  program's  mission,  but 
should  not  provide  direct  support  for 
independent  research. 

NSF  should  increase  the  involvement  of 
NSF's  directorates  in  the  process  of  al- 
locating service  units  at  the  Centers. 

NSF  should  provide  leadership  in 
working  toward  the  development  of  in- 
teragency plans  for  deploying  balanced 
systems  at  the  apex  of  the  computa- 
tional pyramid  and  ensuring  access  to 
these  systems  for  academic  researchers. 

These  recommendations  are  designed  to 
set  the  Centers  program  on  a  new  course 
that  builds  on  its  past  successes,  yet  shifts 
the  focus  to  the  present  realities  of  high- 
performance  computing  and  communica- 
tions, and  provides  flexibility  to  adapt  to 
changmg  circumstances.  It  is  our  expec- 
tation, that  at  current  NSF  budget  levels 
and  absent  new  outside  resources,  there 
will  be  a  reduction  in  the  number  of  lead- 
ing-edge sites  to  effect  the  benefits  of  the 
Task  Force  recommendations. 

In  developing  these  recommendations, 
the  Task  Force  obtained  extensive  input 
from  academic,  government,  and  indus- 
trial leaders:  visited  Centers  and  sought 
written  input  from  the  community. 
Some  of  this  input  is  included  as  appen- 
dices, and  the  complete  input  is  available 
on  the  Internet.  The  issues  are  complex 
and  there  are  many  strongly  held  opin- 
ions on  the  purpose,  execution,  and  value 
of  the  program.  The  Task  Force  has  tried 
to  hear  and  understand  all  of  the  input. 
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but  in  the  end  has,  of  necessity,  formed 
its  own  judgment  of  what  is  best  for  the 
country.  This  report  attempts  to  explain 
that  judgment. 

The  report  begins  with  a  history  of  the 
Centers  and  how  they  fit  into  the  nation's 
high  performance  computing  infrastruc- 
ture. 

The  .second  section  attempts  to  identify 
factors  the  Task  Force  thinks  are  impor- 
tant in  the  evaluation  process,  including 
staff  involvement  in  research,  size  of  the 
u.ser  ba.se,  scientific  discipline  of  the  us- 
ers, funding  leverage,  industrial  partner- 
ships, multidi.sciplinary  activities,  re- 
source availability,  and  education. 

The  "hard  issues"  surrounding  the  Cen- 
ters, particularly  those  not  adequately  dis- 
cussed in  previous  reports,  are  discussed 
in  the  third  section.  This  .section  exam- 
ines such  issues  as:  sunsetting  the  Cen- 
ters: industrial  u.se;  effect  of  "free"  com- 


puter cycles  on  the  market:  the  total  need 
for  high-end  computing;  quality  of  the 
science;  role  of  other  centers;  technology 
and  computer  industry  trends;  and  the 
role  of  the  Centers  in  the  larger  federal 
and  international  context. 

Section  four  examines  five  options  for  a 
Centers  program,  ranging  from  the  cur- 
rent system  to  termination  of  the  pro- 
gram. Other  options  include  partnership 
centers  with  stronger  links  between  lead- 
ership centers  and  university  or  state  fa- 
cilities; a  single  partnership  center;  and 
disciplinary  centers  along  the  lines  of  the 
National  Center  for  Atmospheric  Re- 
.search.  The  pros  and  cons  of  each  option 
are  discussed. 

The  fifth  section  of  the  report  discusses 
future  directions  and  priorities  for  the 
Centers  program.  The  final  section  re- 
states and  explains  each  of  the  seven  spe- 
cific recommendations  designed  to  sup- 
port the  Task  Force  vision  for  the  future. 
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Introduction 

The  NSF  Centers  Program  was  established 
to  provide  access  to  high  pert'ormance  com- 
puters (supercomputers)  tor  the  broad  Sci- 
ence and  Engmeermg  Research  Community. 
The  program  has  evolved  from  one  com- 
prising independent,  competitive.  ar>d  similar 
computer  centers  to  one  mcluding  more  co- 
operative and  diverse  activities.  Coordinating 
the  mission  of  tlie  indi\iduai  centers  has  in- 
creased the  diversity  of  computer  architec- 
tures available  to  the  research  community, 
and  has  accelerated  outreach  to  segments  of 
the  community  which  had  not  before  been 
able  to  use  the  power  of  high  performance 
computers.  At  the  same  time,  competition 
between  centers  has  been  managed  by  NSF 
and  its  advisory  committees  to  the  advantage 
of  the  engineering  and  science  communities 
which  the  program  was  established  to  .serve. 

Building  on  each  center's  tradition  of  pro- 
viding a  stable  source  of  computer  cycles  for 
a  large  community  of  .scientists  and  engi- 
neers, the  centers  have  evolved  into  a  unique 


resource  to  test  which  diverse  computer  ar- 
chitectures best  match  the  most  demanding 
problems  posed  by  the  community  of  uni- 
versity researchers  and  to  develop  the  neces- 
sary supporting  software  and  algorithms.  For 
example,  this  approach  has  enabled  the  cen- 
ters to  test  which  applications  can  be  effi- 
ciently served  on  newly  developed  systems 
using  clusters  of  the  new  generation  of  work- 
stations that  are  now  being  introduced.  Such 
experiments  are  enabled  b\  the  open  en\i- 
ronment  characteristic  of  the  program. 

During  the  first  decade  of  the  centers  pro- 
gram, major  improvements  in  the  delivery  of 
high  performance  computing  have  been  de- 
veloped, mainly  by  American  computer  re- 
searchers and  companies.  But  advances  in 
computing  technology  have  been  matched  in 
equal  measure  by  improvements  in  computer 
networking,  and  as  a  consequence  the  NSF 
Centers  have  been  a  primary  focus  for  accel- 
erating the  evolution  of  the  Internet  via 
NSFnei.  NREN.  and  the  still  evolving  broad- 
band width  technology. 

in  this  appendix,  we  provide  detailed  exam- 
ples of  the  centers  activities. 


Important  Technology  Accomplishments 


High  Performance  Computing 

Originall>  set  up  in  1985  to  provide  national 
access  to  traditional  supercomputers,  the 
NSF  Centers  have  evolved  to  a  much  larger 
mission.  The  Centers  now  offer  a  wide  vari- 
ety of  high  performance  architectures  from  a 
large  array  of  American  vendors.  No  longer 
lUst  adopting  technology  from  the  national 
labs,  the  NSF  Centers  Program  has  become  a 
pioneering  vanguard  of  technology  —  a 
model  for  other  agencies  with  a  vested  inter- 


est in  the  high  performance  computing  to 
emulate.  This  today  is  dominated  by  research 
efforts  in  software,  with  vital  collaborations 
with  computer  scientists,  focusing  on  oper- 
ating systems,  compilers,  network  control, 
mathematical  libraries,  and  programming 
languages  and  environments.  The  feedback  to 
the  leading  US  vendors  is  increasing  the  use- 
fulness of  their  product  offerings  to  the  sci- 
entific and  engineering  communities,  while 
making  them  more  competitive. 
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Supercomputer  Usage  at  NSF  Centers 


Fiscal  Year 

Active  Users 

Usage  in  CPU 
Hours 

1986 

1,336 

29,485 

1987 

3,299 

95,751 

1988 

5,042 

121,615 

1989 

5,967 

165,960 

1990 

7,357 

250,628 

1991 

7,723 

361,073 

1992 

8,252 

398,931 

1993 

7,735 

910,088 

1994 

7,395 

2,370,794 

1995 

6,601 

4,590,606 

(Usage  IS  111  normalized  CPU  hours,  based  on  comparative  performance  tests.  The  astoundmg  leap  m  ca- 
pacity in  1993  is  mamly  a  result  of  the  introduction  of  new  computing  architectures  to  solve  the  most 
demanding  of  computational  problems  — the  Gr2tnd  Challenges.  The  slight  decrease  in  the  number  of 
users  is  the  result  of  a  concerted  effort  by  the  Centers  to  assist  many  of  their  users  with  small  memory  or 
CPU-time  requirements  to  meet  their  computational  needs  by  the  increasingly  powerful  workstations  of 
the  mid-90's.  The  greatest  benefactors  of  the  increase  m  massively  parallel  cycles  are  the  scientists  and 
engmeers  addressmg  the  problems  with  the  greatest  computing  demands) 


Architectures  and  Vendors 

The  national  community  has  been  offered 
access  to  a  wide  and  frequently  updated  set  of 
high  performance  architectures  since  the  begin- 
ning of  the  NSF  Supercomputer  Centers  Pro- 
gram The  current  rate  of  change  of  the  types  of 
architectures,  and  the  number  of  vendors  of- 
i'enng  them,  is  probably  near  an  all  time  high. 
We  are  in  a  period  of  ferment  which  the  sci- 
ence and  engineermg  communities  sort  out  the 
choices  for  finding  an  architecture  that  matches 
theii  various  computational  problems.  A  list  of 
architectures  that  the  NSF  Centers  Program  has 
offered   would   include:   single    and   clustered 


high  performance  workstations  or  workstation 
multiprocessors,  minicomputers,  graphics 
supercomputers,  mainframes  with  or  without 
attached  processors  or  vector  units,  vector 
supercomputers,  and  SIMD  and  MIMD  mas- 
sively parallel  processors.  Similarly,  the  list  of 
current  vendors  whose  top  machines  have  been 
made  available  would  include  IBM,  DEC, 
Hewlett-Packard,  Silicon  Graphics,  Sun  Mi- 
crosystems, Cray  Research,  Convex  Com- 
puter, Intel  Supercomputer,  Kendall  Square. 
Thinking  Machines.  nCUBE.  Aliiant,  Floating 
Point  Systems.  ETA.  Stellar,  Ardent,  and 
Slardent. 
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Center  Program  Chronology 


f\ 

Milestone  or  Event 

Description 

1986 

?  NSF  Superaimputer  Centers  become 
Dperational 

Cornell  Theory  Center 

National  Center  tor  Supercomputing  Applications 

Pittsburgh  Supercomputing  Center 

San  Diego  Superconiputer  Center 

lohn  von  Neumann  Center 

lyss-iywy 

Renewal  Re\  iew 

1989 

Renew  4  NSF  Supercomputer  Centers 

CTC,  NCSA,  PSC,  SDSC 

1990-1492 

ASL  Ad\isor\-  Committee  report  com- 
pleted 

strong  recommendations  tor  adding  parallel  systems  to 
accompan\  the  stable,  production  vector  svstems 

1991 

,s  \  ector  Supercomputers  operatmt; 
3  Scalable  parallel  systems  operatmg 

IBM  3090                   h  processors  (2) 
Cray  YMP                8  processors  (2) 
Alh'ant  FX80            S  processors 
Cray  2S                     4  processors 
Cray  YMP                4  processors 
Con\'ex  C240           4  processors 
Intel  iPSC/860              32  processors''' 
TMC,  CM2                     32,000  processors  (2|'^ 

1992 

7  Vector  Supercomputers  operatmg 
9  Scalable  parallel  systems  operating 

lomt  Plannmg  Initiated 

Cray  s\'stems  remain 

IBM  ES9000/900  (Upgrade  3090) 

Alhant  2800  (upgrade  FX80) 

Con\'e\  C3880         8  processors  (Upgrade  C240) 
Intel  iPSC/860  upgrade       64  processors)! 
nCUBE2                         128  processorst 
TMC,  CM5                    512  processorst 
KSRl                             64  processorst 
DEC  Workstation  Cluster  (2)t 
IBM  Workstation  Cluster 
IBM  PVS                        32  processors 

Initial  meeting  at  SDSC  Fall  92 

1993 

Joint  Activities  Began 

hVector  Supercomputers  operating 
13  Scalable  parallel  systems  operating 

Meeting  at  PSC 
First  |omt  proposals  to  PPRP 
First  of  Joint  projects 
Cray  C90                  16  processors 
(other  Crays,  IBM,  Convex  stay) 
Intel  Paragon                 400  processors  upgrade  t 
KSRl  upgrade                160  processorst 
Hewlett  Packard  Cluster^ 
MasPar  2                        16,000  processors^ 
IBM  SPl                          64  processorst 
Cra\  T3D                       5]2  processorst 

1994 

lomt  Activities 

4  Vector  Supercomputers  operating 
14  Scalable  parallel  systems  operating 

.Meeting  at  CTC 
Metacenter  Regional  Alliances  -  Mar  94 
Expansion  of  |oint  proiects 
One  YMP  changed  to  C-90,  others  the  same 
IBM  SP2  upgrade          512  processors  upgrade  SPlt 
Cray  T3D                       512  processors  t 
Convex  Exemplar          8  Nodest 
SGI  Challenge                32  Nodes 

^  Ma|ority  of  funding  provided  by  other  Federal  agency  (ARPA,  NIH)  or  state. 
i  Donated  m  full  or  in  part  by  the  manufacturer  for  extended  evaluation 
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The  Centers  Prdgram  provides  a  stable  sup- 
ply of  vector  computing  cycles  needed  by  the 
research  community  while  investmg  in  sig- 
nificant capacity  of  scalable  parallel  systems, 
capable  of  ultimately  growing  to  a  size  neces- 
sary for  full  scale  grand  challenge  problems. 
While  the  numbers  of  Vector  Supercom- 
puters has  decreased,  the  computing  power 
represented  by  that  group  in  fact  increased 
substantially.  However,  the  increase  in  the 
scalable  parallel  systems  was  much  larger, 
reflecting  the  growth  potential  of  this  type  of 
computing  platform  and  a  strong 
NSF/ARPA  partnership. 

National  Access  to  Vector  Multiproc- 
essors 

The  NSF  Supercomputer  Centers  established 
in  the  mid-1980s  brought  access  to  state-of- 
the-art  supercomputers  for  the  first  time  in  at 
least  15  years.  Indeed,  in  the  1960s,  only  a 
few  universities  had  such  access.  This  open- 
ing of  universal  access  led  to  an  unprece- 
dented increase  in  the  number  of  researchers 
and  universities  involved  in  advancing  the 
frontiers  of  scientific  and  engineering  research 
by  using  high  performance  computing.  By  the 
early  1990s,  some  15,000  researchers  in  over 
200  universities  had  u.sed  one  of  the  Cray 
Research  vector  multiprocessors  or  the  IBM 
vector  mainframe  in  one  of  the  NSF  centers. 
This  wide  pool  of  computational  researchers 
made  it  possible  for  the  center's  program  to 
begin  to  respond  to  the  demand  for  paral- 
lelism that  had  been  developed  in  the  Com- 
puter Science  Community,  and  adopted  by  the 
most  adventurous  user.  The  90" s  saw  the 
NSF  center's  program  substantially  widen  its 
range  of  architectural  offerings. 

Achieving  Production  Parallelism 

The  Cornell  Theory  Center  (CTC)  became  the 
first  member  Center  of  the  NSF  MetaCenter 
to  achieve  production  parallelism  on  a  vector 
supercomputer,  with  over  1/3  of  its  vector 
supercomputer  cycles  used  for  parallelism  in 
1989.  CTC  integrated  its  two  (ES/3090  600) 
vector  supercomputers  using  a  special  200 
mbyte/sec  hardware  interface,  allowing  paral- 
lel jobs  the  potential  of  executing  across    12 


vector  processors.  Users  u.sed  a  shared- 
memory  parallel  FORTRAN  developed  by 
IBM  in  a  joint  project  with  the  Theory  Center. 

Early  Migration  to  the  UNIX  Operat- 
ing System 

During  the  early  and  mid-1980's  the  UNIX 
operating  system  was  widely  viewed  as  inap- 
propriate for  supercomputers  for  reasons  of 
performance,  system  management  tools,  ap- 
plication development  and  measurement 
mechanisms,  and  security.  Cray  supercom- 
puters were  run  mostly  with  operating  sys- 
tems that  were  designed  at  national  laborato- 
ries (LLNL  and  LANL  in  particular)  and  this 
required  extensive  local  software  support.  In 
1987,  NCSA  became  the  first  major  super- 
computer center  to  migrate  its  Cray  super- 
computer from  CTSS  (Cray's  proprietary 
time-.sharing  system)  to  UNICOS,  a  UNIX- 
based  operating  system  developed  at  Cray 
Research  for  its  supercomputers.  This  move 
to  UNIX  was  the  beginning  of  a  merger  be- 
tween computational  science  and  computer 
science,  because  most  computer  .science  re- 
search involved  the  UNIX  operating  system  at 
that  time.  Coincidentally,  CTC  was  the  first 
site  to  run  IBM's  high  performance  UNIX 
system  on  its  ES/3090  and  ES/9000  main- 
frames in  production. 

Early  Access  to  Massively  Parallel 
Computers 

Beginning  in  1985,  CTC  provided  ex- 
perimental scalable  parallel  machines,  first 
the  EPS  T-series  and  an  iPSC/l  parallel  sys- 
tem, to  its  user  community.  In  1988.  with  the 
installation  of  its  iPSC/2  with  32  processors, 
CTC  made  early  .scalable  computing  available 
for  production  use  by  the  national  com- 
munity. Massively  parallel  computing  was 
introduced  to  the  re.search  community  begin- 
ning with  NCSA's  CM-2  in  1989.  Each 
Center  provided  early  access  to  new  genera- 
tion MPPs.  The  CTC  was  the  first  site  to  in- 
stall an  IBM  SPI  and  SP2.  PSC  installed  the 
first  Cray  T3D,  and  early  CMS  (NCSA), 
Paragon(SDSC),  Ncube(SDSC),  and  Ken- 
dall Square  (CTC)  machines  were  installed. 
Early  access  to  these  machines,  enabled  by 


-4- 


51 


High  Performance  Computing  Infrastructure  and  Accomplishments 


support  from  ARPA  and  NIH,  allowed  pio- 
neering users  to  explore  the  benefits  of  fine- 
grained parallelism.  Each  Center  worked 
with  the  user  community  and  the  vendors  to 
develop  application  codes,  which  could  then 
be  ported  to  other  platforms.  In  1992,  the 
CM-5  was  added  to  the  program  at  NCSA  as 
the  largest  distributed  memory  parallel  super- 
computer available  to  the  national  academic 
and  industrial  communities.  From  1992  to 
the  present  NCSA  has  worked  closely  with 
national  users  and  the  computer  .science 
community  to  create  a  wide  range  of  512- 
way  parallel  application  codes  that  can  in 
1995  be  moved  to  other  large  MPP  architec- 
tures such  as  the  T3D  at  PSC.  the  Intel  Para- 
gon at  SDSC.  or  the  IBM  SP-2  at  CTC. 

Superhnear  speedup  on  heterogeneous 
processors 

In  1991.  PSC  was  the  first  site  to  distribute 
code  between  a  massively  parallel  machine 
(TMC-CM2)  and  a  vector  supercomputer 
(Cray  YMP).  linked  by  a  high  speed  channel 
(HiPPI).  Experiments  on  applications  as  di- 
verse as  molecular  dynamics,  medical  imag- 
ing, chemical  flowsheeting  and  gene  sequence 
alignment  showed  superhnear  speedup  (the 
applications  on  the  linked  system  ran  more 
than  twice  as  fast  on  each  system  separately). 
This  formed  part  of  the  motivation  for  het- 
erogeneous computing,  as  later  embodied  in 
the  tightly  coupled  Cray  T3D/C90  systems. 
PSC's  T3D  was  the  first  shipped  anywhere. 
PSC  developed  a  set  of  codes  for  transferring 
data  between  the  Cray  and  CM2  which  later 
enabled  them  to  communicate  between  the 
T3D  and  C90  at  speeds  superior  to  what  was 
available  from  the  vendor. 

A  similar  superhnear  speedup  was  obtained 
on  the  CASA  gigabit  testbed  set-up  included 
parts  of  two  supercomputers  (64  nodes  of  the 
528-node  Intel  Delta  svstem  at  Caltech  and 
one  processor  of  the  CRAY  C-90  at  SDSC. 
150  miles  away);  two  HiPPI-SONET  gate- 
ways; and  a  SONET  wide-area  link  between 
San  Diego  and  Pasadena,  which  is  itself  a 
prototype  undergoing  tests  in  a  collaboration 
between  MCI  and  Pacific  Bell.  Usine  an  en- 


vironment for  distributed  parallel  computing 
called  Express  (a  product  of  ParaSoft  Corpo- 
ration). Aron  Kuppermann  and  Mark  Wu 
(Dept.  of  Chemistry,  CalTech)  did  a  test  cal- 
culation of  the  reaction  of  atomic  hydrogen 
with  molecular  heavy  hydrogen  (deuterium) 
at  a  total  energy  of  2.5  eV.  This  problem  had 
taken  100  hours  to  .solve  on  the  SDSC 
CRAY  Y-MP  a  year  earlier.  On  the  new  C- 
90,  It  took  17  hours.  On  the  Delta  alone,  it 
took  16.  But  when  the  problem  was  distrib- 
uted between  the  C-90  and  the  Delta,  the 
whole  problem  was  solved  by  the  two  ma- 
chines in  just  under  five  hours,  a  factor  of  3.3 
faster  than  it  could  be  done  on  either  machine 
alone. 

Workstation  Clusters 

Given  historical  trends  showing  much  more 
rapid  improvement  in  microprocessor  tech- 
nology than  in  vector  technology,  many  Cen- 
ters began  exploring  workstation  clusters,  in 

1990.  NCSA  examined  the  usage  of  the  Cray 
Y-MP  and  determined  that  a  significant 
amount  of  capacity  could  be  gained  by  mov- 
ing appropriate  applications  to  scalar  RISC 
processors.  Based  on  this  study  and  on  pre- 
dictions that  microprocessor  technology 
would  surpass  vector  technology  by  the  mid- 
1990's.  NCSA  set  up  an  IBM  RS/6000  clus- 
ter as  a  farm  of  processors  used  as  stand- 
alone compute  servers.  Beginning  in  1991, 
CTC  did  pioneering  work  with  IBM  on  clus- 
tered RS/6000  workstations  with  high-speed, 
proprietary  communications  links,  including 
an  experimental  optical  switch.  The  informa- 
tion gained  during  this  joint  project  was  used 
to  guide  the  design  of  the  SP  systems.  Also  in 

1991.  PSC  set  up  a  cluster  of  DEC  worksta- 
tions, and  working  with  Florida  State  Univer- 
sity, significantly  enhanced  queuing,  account- 
ing and  control  software  and  also  integrated 
the  AFS  file  system  into  this  environment. 
These  clusters  all  served  as  computing  re- 
sources and  al.so  as  platforms  for  the  devel- 
opment of  distributed  memory  message 
passing  codes.  These  projects  generated  high 
interest  in  industry,  and  NCSA  trained  several 
dozen  industrial  sites  on  the  integration,  op- 
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eration.  and  management  of  clusters  ot  mi- 
croprocessors. NCSA  is  now  moving  to  es- 
tablish clusters  of  shared  memory  workstation 
multiprocessors  from  Convex/HP  and  SGI, 
while  PSC  will  develop  applications  for 
Cray's  new  offering,  the  J90.  The  CTC  was 
given  supplemental  funding  by  NSF  to  build 
its  cluster  to  32  processors,  providing  a  com- 
patible path  to  Its  (later)  IBM  SPl  and  SP2 
systems.  The  system,  including  experimental 
high-speed  switch,  was  used  for  production 
work  by  the  community,  which  has  now  mi- 
grated to  the  SPl  and  SP2  environments. 

Portable  Parallel  Programming 
Tools 

Although  the  architectures  of  massively  par- 
allel systems  differ  greatly,  the  major  time 
and  money  investment  of  the  research  com- 
munity (as  contra-sted  to  the  center's  person- 
nel) is  in  developing  and  converting  codes 
(porting)  to  operate  in  different  envi- 
ronments. The  close  cooperation  between  the 
NSF  Centers  via  the  MetaCenter  and  infor- 
mal contacts  among  its  research  users  and 
cooperating  agencies  such  as  ARPA  and  the 
various  National  Labs  have  resulted  in  sub- 
stantial progress  ensuring  that  labor  intensive 
programming  operations  need  not  be  dupli- 
cated needlessly. 

Prototype  Parallel  Programming  En- 
vironments 

Working  with  the  Parascope  Group  at  the 
Center  for  Research  in  Parallel  Computing,  an 
NSF  Science  and  Technology  Center,  CTC 
developed  extensions  supporting  new  parallel 
programming  paradigms  and  extensions 
making  ports  from  one  type  of  parallel  pro- 
gramming platform  to  another  easier.  ParaS- 
cope  is  a  prototype  parallel  programming  en- 
vironment. Both  the  BIMA  and  Cosmology 
GCs  at  NCSA  are  working  closely  with  Indi- 
ana University  Computer  Scientist  Dennis 
Gannon  to  move  application  codes  previously 
written  in  FORTRAN  to  the  portable  pC-n- 
which  IS  the  model  for  HPC-t-i-,  the  equivalent 
of  HPF  in  the  C-i-i-  world. 


Extensions  of  PVM 

In  the  Dome  project,  Adam  Beguelin,  one  of 
the  original  developers  of  PVM  now  working 
jointly  at  PSC  and  in  Computer  Science  at 
Carnegie  Mellon  University,  is  extending 
PVM  to  improve  load  balancing  and  fault 
tolerance.  His  work  is  guided  by  the  experi- 
ence of  PSC's  cluster  users. 

Scalable  Libraries 

The  goal  of  the  ARPA  funded  Scalable  Paral- 
lel Libraries  project  is  to  develop  mathematical 
software  libraries  for  massively  parallel  proc- 
es.sors  that  are  roughly  comparable  in  scope  to 
the  math  libraries  typically  available  on  con- 
ventional supercomputers.  Michael  Heath  and 
his  group  at  NCSA  are  one  team  in  this  multi- 
institutional  project  and  they  have  been  devel- 
oping parallel  direct  methods  for  solving 
sparse  linear  systems.  For  this  purpose,  they 
have  developed  a  fully  parallel  sparse  solver 
for  distributed  memory  parallel  computers. 
Unlike  most  other  efforts,  which  have  focused 
only  on  factorization,  this  solver  performs  all 
phases  of  the  computation  in  parallel  mode, 
including  the  symbolic  preprocessing  neces- 
sary to  reorder  the  sparse  matrix  and  distribute 
it  across  processors  to  maintain  data  locality. 
With  funding  from  IBM,  CTC  staff  devel- 
oped scalable  versions  of  key  numerical  li- 
brary routines  for  its  IBM  cluster  system: 
these  routines  were  included  by  IBM  in  its 
ESSL  library  product. 

Storage  Technologies 

With  the  vast  increase  in  both  simulation  and 
observational  data,  the  MetaCenter  has 
worked  a  great  deal  on  problems  of  storage 
technologies.  Here  again,  many  of  the  biggest 
areas  of  progress  are  in  software.  The  crea- 
tion of  a  universal  file  format  standard,  a  na- 
tional file  system  with  a  single  name  space, 
and  a  multivendor  archiving  software  are 
some  of  the  results  of  MetaCenter  innova- 
tion, collaboration  with  computer  scientists, 
and  with  other  leading  national  laboratories. 
There  are  even  examples  of  the  Program's 
computational   facilities  being   used   to   im- 
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prove  the  basic  storage  capacity  of  the  physi- 
cal medium  ot  storage  itself. 

AFS-Estabiishing  a  National  File  Sys- 
tem 

PSC  recognized  that  the  Andrew  File  System 
(AFS).  developed  at  Carnegie  Mellon  Uni- 
versity with  IBM  support  for  a  workstation 
environment,  was  particularly-well  suited  for 
use  in  high  performance  computing,  because 
of  its  superior  security,  .scaling  properties, 
and  manageability.  PSC  undertook  a  major 
program  of  adapting  AFS  to  the  high  per- 
formance computing  environment  which  has 
led  to  a  MetaCenter  wide  effort  to  develop  a 
shared  national  file  system.  PSC's  AFS  en- 
hancements to  Cray's  UNICOS  are  installed 
at  a  number  of  advanced  computing  centers 
(SDSC.  .NERSC.  IPP.  LRZ  (Germany). 
ETH  (Switzerland)  and  the  University  of 
Stuttgart).  PSC  has  also  extended  AFS  to 
multi-resident  AFS  which  enables  AFS  to  be 
a  component  of  a  hierarchical  storage  system. 
permitting  transparent  and  cost-effective  stor- 
age of  large  amounts  of  data.  These  en- 
hancement are  installed  at  other  MetaCenter 
sites.  NERSC.  Cray  Research.  Transarc  Cor- 
poration. Ma.x  Planck  Institute  for  Plasma 
Physics,  and  the  University  of  Cologne.  With 
AFS.  distributed  applications  across  the 
MetaCenter  are  now  possible.  For  instance. 
Paul  Dawson  (Dept.  of  Mech.  and  Aerospace 
Engineering.  Cornell  Univ.).  simulating  de- 
formations of  aluminum,  uses  AFS  to  build 
a  distributed  application  with  modeling  per- 
formed at  PSC  and  visualization  at  the  CTC. 
The  SDSC  is  running  an  AFS  cell  supporting 
the  Computational  Center  for  Macromolecu- 
lar  Structure  (CCMS).  an  SDSC.  UCSD.  and 
Scripps  Research  Institute  collaboration.  The 
charier  of  the  CCMS  is  the  development  and 
distribution  of  portable,  innovative  sotfware 
for  the  study  of  macromolecular  structure. 
AFS  simplifies  the  cross-institution  distribu- 
tion and  maintenance  of  software  and  textual 
information. 

CTC  was  the  first  Center  to  put  AFS  in  pro- 
duction on  its  HPC  systems;  in  fact,  all  CTC 
production  systems,  except  KSR.  and  servers 


are  integrated  through  AFS.  including  its  new 
mass  storage  environment.  CTC  has  lead  in  a 
number  of  areas  of  file  system  integration, 
including  a  project  with  the  University  of 
Michigan  integrating  its  AFS  mainframe 
port.  IFS.  into  its  HPC  environment,  and 
joint  efforts  with  TRANSARC  in  improving 
AFS  performance  over  high  speed  networks, 
including  an  FDDI  testbed.  New  efforts  in- 
volving TRANSARC  include  optimization 
over  ATM  and  within  IBM's  SP2  scalable 
switch. 

HDF-Creating  a  Standard  File  For- 
mat 

The  NCSA  Hierarchical  Data  Format  (HDF). 
created  by  Michael  Folk  and  his  group  at 
NCSA  has  become  one  of  the  leading  self- 
describing  file  formats  in  the  world  today. 
Many  scientific  institutions,  organizations  and 
programs  have  adopted  HDF  as  a  standard 
file  format  for  data  exchange  and/or  archiving. 
In  1992.  NASA  selected  ^HDF  as  the  basis 
from  which  to  develop  an  EOSDIS  standard 
data  format  (SDF).  The  goal  of  SDF  is  to 
provide  a  single,  self-describing  format  for 
distributing  data  derived  from  approximately 
1.2  terabytes  of  data  daily  that  EOSDIS  will 
eventually  produce.  Other  examples  of  HDF 
adoption  include  the  Institute  of  Applied 
GeoScience  (seismic  data).  Pacific  Northwest 
Laboratory  (cancer  research).  Children's  Hos- 
pital in  Boston  (x-ray  images),  the  UCLA 
Scientific  Visualization  Lab.  By  working 
closely  with  many  different  user  communities 
to  support  the  harmonization  of  data  models 
and  metadata  conventions  across  as  many 
disciplines  as  possible.  NCSA  is  helping  to 
create  a  software  foundation  for  the  Nil  en- 
abling It  to  reach  its  potential  to  support  the 
broadest  constituency  possible. 

Migrating  to  a  Standard  Archiving 
Software 

In  1985.  the  NSF  Centers'  goal  was  simply 
to  establish  national  access  for  the  academic 
community  to  the  type  of  advanced  super- 
computing  and  archiving  systems  found  in 
the  Dept.  of  Energy  national  laboratories. 
NCSA  and  SDSC  "duplicated  the  Los  Ala- 
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mos  computing  environment  in  1985,  includ- 
ing the  Common  File  System  (CFS)  archive, 
while  PSC   adopted  Westinghouse's   PDM 
software  that  had  additional  data  migration 
facilities.  As  time  went  on,  the  Centers  began 
to  develop  innovation  in  storage  software.  It 
is  the  strategy  of  the  MetaCenter  to  explore 
alternative  technical  approaches  to  storage  at 
the    bits    level,    while    maintaining    interop- 
erability through  standard  protocols.  SDSC. 
through  the  DISCOS  (Distributed  Computer 
Solutions)    division    of    General     Atomics 
(which  is  now  owned  by  Open  Vision),  pio- 
neered productization  of  distributed,  hierar- 
chical   file    and    storage    management    ap- 
plications  software    for    networked,    multi- 
vendor    and    open    sy.stems    environments, 
based  on  the  IEEE  storage  model.  Working 
with    the    UIUC    Computer    Science    De- 
partment, NCSA  was  able  to  encode  data 
migration  and  caching  strategies  into  CFS  to 
improve  its  ability  to  minimize  disk  cache 
misses.  In  1992,  NCSA  developed  CFS-to- 
UniTree  data  formatting  and  migration  tools 
as  well  as  a  suite  of  archive  management 
tools.  PSC  integrated  Cray's  proven  com- 
mercial  archiving   technology   (DMF)    with 
more  usable  front-end  software  and  with  its 
multi-resident  AFS.  SDSC  collaborates  with 
OpenVision  and  the  National  Storage  Labo- 
ratory in  the  further  development  and  stabili- 
zation of  UniTree  as  a  robust  production  ar- 
chival storage  system.  SDSC  has  developed 
transaction  journaling  software  that  is  criti- 
cally important  for  guaranteeing  integrity  of 
the  file  name.server.  CTC  and  SDSC  are  in- 
.stalling    the    first    generation    of    a    high- 
performance  variant  of  UNITREE  which  has 
been    developed    by    the    National    Storage 
Laboratory,  to  provide  enhanced  L/O  capabil- 
ity to  balance  the  increa.se  in  in.stalled  com- 
puting capacity.  CTC  will  continue  to  work 
closeFy  with  IBM  in  the  testing  and  deploy- 
ment of  the  next  generation  of  mass  storage 
systems,    the    High    Performance    Storage 
System  (HPSSl.  This  will  include  the  ca- 
pability of  utilizing  parallel  VO  to  speed  data 
transfer  to  the  SP2. 


Development  of  high-density  magnetic 
media 

The  Center  for  Magnetic  Recording  Research 
(CMRR),  located  on  the  campus  of  UCSD. 
has  funding  from  NSF  and  21  corporations 
having  enterprises  connected  with  the  storage 
and  retrieval  of  magnetically  written   infor- 
mation. The  chief  technical  problem  CMRR 
has  been  addressing  has  been  magnetic  noise 
in  the  metallic  thin-film  media  used  to  coat 
high-density  disks.  Neal  Bertram  (Depi.  of 
Electrical      and      Computer      Engineering, 
UCSD)  is  a  researcher  at  CMRR  who  tackles 
the  problem    of   magnetic    noise   computa- 
tionally.   Magnetic    thin    films     are     poly- 
cry.stalline.  rather  than  continuous  or  amor- 
phous, and  the  grainy,  particulate  nature  of 
the  medium  is  a  fundamental  source  of  noise. 
Bertram's  calculations  have  explored  the  ef- 
fects of  two  primary  types  of  interaction  be- 
tween grains  that  cause  noise:  magnetostatic 
coupling  and  exchange  coupling.  The  calcu- 
lations resulted  in  recommendations  for  al- 
loys and  fabrication  processes  that  would  re- 
duce noise  from  both  sources.  The  research 
enabled  engineers  at  IBM's   Almaden   Re- 
search Center  to  design  a  disk  coating  that 
packs   a   gigabit    (one    billion    bits)    of    in- 
formation onto  a  square  inch,  which  is  15-30 
times  current  storage  densities.  Bertram  has 
now  turned  his  attention  to  calculations  ot 
other  effects,  including  giant  magnetoresis- 
tance,  that  can  be   important    in    designing 
high-density  disk  media  and  disk  recording 
and  playback  heads.  His   codes   model   the 
process  of  recording  bits  in  intricate  detail; 
the  process  of  laying  down  a  single  bit  of  in- 
formation takes  several  minutes  to  calculate 
on  SDSC's  Cray  Research  C90. 

Networking 

One  of  the  great  succes.ses  of  the  NSF  Meta- 
Center has  been  in  providing  the  "high-end 
pull"  that  has  led  to  the  creation  and  expo- 
nential evolution  of  the  NSFnet.  As  a  result, 
the  NSFnet  backbone  of  1995  has  3000 
times  the  bandwidth  of  the  backbone  of 
1986.  The  Centers  have  also  prototyped  the 
high  performance  local  area  networks  that  are 
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needed  to  feed  into  the  national  backbone  as 
well  as  the  next  generation  of  gigabit  back- 
bones. Security  over  networks  is  essential  not 
only  for  industrial  usage,  but  more  and  more 
for  widespread  citizen  usage.  Again,  the 
MetaCenter  has  created  innovations  such  as 
dealing  with  mdustrial  firewalls.  The  exis- 
tence of  the  MetaCenter  network  testbeds  al- 
lows for  new  kinds  of  science  to  be  attacked, 
perhaps  best  illustrated  by  the  rise  of  telemi- 
croscop),  m  which  leading  edge  projects  are 
being  carried  out  at  each  of  the  NSF  Super- 
computer Centers.  With  the  national  priority 
on  the  .Ml,  the  Centers  are  moving  rapidly  to 
expand  their  networking  research  to  commu- 
nity based  Nil  lestbeds.  including  local 
healthcare,  education,  government,  and  small 
business  partners. 

Evolution  of  NSFNET 

The  56kbps  connection  between  the  NSF 
Centers,  established  in  1986,  was  the  begin- 
ning of  the  NSFnet.  Based  on  the  successes  of 
ARPAnet  and  the  TCP/IP  protocol  within  the 
computer  science  and  Dept.  of  Defense  com- 
munities, the  NSFnet  rapidly  grew  to  provide 
remote  access  to  the  NSF  Supercomputer 
Centers  by  the  creation  of  regional  and  cam- 
pus connections  to  the  backbone.  Although 
started  by  the  pull  from  the  high  end,  the 
NSFnet  soon  began  to  provide  ubiquitous 
connectivity  to  the  academic  research  commu- 
nity for  electronic  mail,  file  transport,  and 
remote  login,  as  well  as  supercomputer  con- 
nectivity. These  daily  uses  soon  became  in- 
di.spensable  to  the  research  community  and 
the  sustained  exponential  growth  of  the  In- 
temet  took  off.  The  MetaCenter" s  industrial 
partner  network  was  among  the  first  in  the 
coiporate  world  to  use  NSFnet/lnternet  tech- 
nology to  connect  corporations  to  the  Internet 
for  the  purposes  of  computational  science. 
This  was  an  important  precursor  for  today's 
rapid  commercialization  of  the  Internet. 

By  early  1995,  the  NSFnet  will  return  to  a 
high  speed  backbone  connecting  the  Meta- 
Center and  some  of  the  newly  selected  Meta- 
Center Regional  Alliance  members.  However, 
the  bandwidth  of  the  backbone  will  be  3000 


times  higher  than  that  of  the  original  backbone 
(56  kbps).  While  .some  tend  to  think  of  the 
MetaCenter  as  focusing  on  high  performance 
computing  only,  it  is  useful  to  remember  that 
computing  power  of  the  fastest  supercom- 
puter/?n«c'.v.sr.<;- in  the  program  has  grown  by 
little  more  than  100  times  during  the  same 
period.  Indeed,  it  is  likely  that  the  155  mbps 
vBNS  will  be  upgraded  to  622  mbps  nirliin 
two  years.  Even  by  the  time  the  Centers  re- 
ceive the  first  teraflop  machines  in  1997-98, 
realizing  a  factor  of  1000  increase  in  speed 
over  1985.  the  backbone  will  have  grown  by  a 
factor  of  at  least  25,000  fold  in  bandwidth. 

As  part  of  an  SDSC/UCSD  collaboration, 
Kimberly  Claffy  recently  completed  a  Ph.D. 
dissertation  that  outlined  a  methodology  for 
profiling  Internet  traffic  flows  at  a  variety  of 
granularities.  The  methodologies  and  models 
developed  as  part  of  this  traffic  characteriza- 
tion effort  should  prove  very  useful  as  the 
Internet  evolves  to  an  even  larger  system  in 
which  the  traffic  composition  needs  to  be  un- 
derstood, particularly  for  planning  future 
technology  and  capacity. 

High  Performance  LANs 

The  center's  program  has  also  pioneered  sev- 
eral transitions  in  local  area  and  metropolitan 
area  networks  both  on  site  and  on  university 
campuses,  acting  as  a  prototyping  facility  for 
other  campuses  who  needed  to  know  how  to 
develop  long  range  networking  plans  for  their 
campuses.  In  19S8,  NCSA  in.stalled  the  first 
Ultranet  Gigabit  LAN  networks  with  multiple 
supercomputers  and  demonstrated  480  Mbil/s 
between  the  CRAY-2  and  Cray  Y-MP  su- 
percomputers. In  1989,  NCSA  replaced  tradi- 
tional HYPERchannel  backbone  networks 
with  the  then-emerging  100  Mbit/s  FDDI 
standard.  In  1991.  PSC  began  its  move  to  a 
HIPPI-based  interconnect  between  its  major 
systems. 

In  1993,  NCSA,  several  industrial  partners, 
and  the  UIUC  Computer  Science  Department 
established  a  local  area  ATM  network  testbed 
to  help  corporations  gain  hands-on  experience 
with  ATM  switches  and  interfaces.  Insight 
from  this  ATM  testbed  has  already  been  used 
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to  develop  long-range  corporate  network 
strategies  for  J. P.  Morgan,  Phillips  Petroleum. 
FMC  Corporation,  and  United  Technologies. 

CTC  was  the  first  site  to  integrate  ATM  into  a 
parallel  supercomputer  environment  on  its 
IBM  SPl  in  April  1994  and  its  IBM  SP2  in 
July.  ATM  will  be  u.sed  for  AFS-based  file 
service  and  other  high  speed  transport  needs, 
including  distributed  applications  and  image 
transport. 

Gigabit  Testbeds 

Since  1987.  NCSA  and  the  University  of  Illi- 
nois Computer  Science  Department  have 
worked  with  AT&T  on  the  XUNET  research 
network  testbed  with  capacity  that  is  one  step 
beyond  what  is  available  on  the  Internet. 
While  the  NSFnet  has  moved  from  56  Kbs 
through  1.5  Mbps  to  45  Mbs,  XUNET  has 
moved  from  1.5  Mbs  through  45  Mbs  to  622 
Mbs.  In  July  1993,  XUNET  was  upgraded  to 
622  Mbs.  the  first  network  testbed  to  intercon- 
nect ATM  switches  using  622  Mbs  transmis- 
sion technology  over  long  (>50  miles)  dis- 
tance using  pure  optical  fibers  with  in-line 
optical  amplifiers. 

PSC  has  worked  with  CMU's  Computer  Sci- 
ence Department  in  the  Nectar  Metropolitan 
area  gigabit  testbed  to  develop  new  network- 
ing technology  for  very  high-speed,  low- 
latency  multi-machine  interconnects  and  to 
develop  the  applications  base  which  can  bene- 
fit from  such  technology.  This  work  is  fully 
collaborative  with  PSC's  ground-breaking 
systems  and  applications  level  work  in  het- 
erogeneous systems.  As  a  result  of  testbed 
work,  numerous  applications  can  now  run 
routinely  between  advanced  machines  at 
PSC's  main  hardware  facility  and  tho.se  on  the 
CMU  campus,  15  miles  distant,  at  speeds  of 
up  to  1  Gbit/s. 

In  partnership  with  NYNEX,  Syracu.se  and 
Rome  Laboratory,  now  extended  to  Colum- 
bia, SUNY  Stonybrook  and  Polytechnic  In- 
stitute. CTC  participated  in  building  a  produc- 
tion-level ATM  network  focused  on  demon- 
strating research  and  commercial  applications. 
This  network  was  demonstrated  to  the  Gover- 


nor of  New  York  in  January  1994.  NYNET 
is  also  a  Nil  te.stbed.  involving  outreach, 
medical  applications,  video  on  demand,  as 
described  in  a  later  section. 

Various  applications  are  being  tested  in  the 
CASA  te.stbed.  in  which  SDSC  is  a  major 
partner.  Besides  the  chemical  reaction  dy- 
namics, led  by  Aron  Kuppermann  of  Caltech 
and  mentioned  above  in  the  section  on  super- 
linear  speedup,  there  is  a  coupled  atmo- 
sphere/ocean model  developed  by  the  group 
led  by  Roberto  Mechoso  at  UCLA.  Another 
is  Calcrust,  a  project  directed  by  JPL.  which 
has  u.sed  distributed  heterogeneous  comput- 
ing over  the  CASA  links  to  combine  satellite 
imaging,  .seismic  data,  and  surface  topogra- 
phy in  visualizing  the  foci  of  aftershocks  of 
the  1992  Landers  earthquake  in  Southern 
California. 

Secure  Networks 

In  1987  NCSA  installed  a  1.5  Mbit/s  DS-I 
connection  to  Eastman  Kodak  in  Rochester, 
New  York,  followed  by  another  DS-1  con- 
nection to  Amoco  laboratories  in  Chicago  and 
Tulsa.  By  1990  NCSA  had  connected  over  a 
dozen  indu.strial  laboratories  to  the  Internet  us- 
ing a  combination  of  innovative  security  pre- 
cautions. The.se  included  various  forms  of 
"firewalls"  which  have  now  become  com- 
monplace on  the  Internet. 

New  Science  Enabled  by  Networks— 
Telemicroscopy 

The  San  Diego  Microscopy  and  Imaging  Re- 
source (SDMIR),  led  by  UCSD  neurocy- 
tologist  Mark  Ellisman.  is  an  NIH-funded 
Research  Resource  centered  on  a  new,  fully 
computerized  Intermediate  Voltage  Electron 
Micro.scope  (IVEM).  The  IVEM  is  used  to 
look  at  comparatively  thick  tissue  sections  (2- 
10  microns)  and  it  has  been  employed  in 
studies  of  cortical  neurons  with  and  without 
symptoms  of  Alzheimer's,  in  .studies  of  an- 
other type  of  brain  cell,  called  a  Purkinje  neu- 
ron, and  studies  of  cell  membranes.  A  long- 
term  collaboration  between  SDMIR  and 
SDSC  has  made  the  microscope  usable  inter- 
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actively,   over   the   Internet,   coupled   to    the 
SDSC  Cray  Research  C-90. 

Computational  analysis  and  simulation  is  al- 
lowing biomedical  researchers  to  study  and 
predict  the  activity  of  potential  new  drugs  at 
the  molecular  level.  CTC  is  working  jointly 
v\  ith  Steven  Ealick.  et  al..  director  of  Mac- 
CHESS.  a  group  using  the  Cornell  High  En- 
ergy Synchrotron  Source  for  Macromolecular 
Modeling.  Using  existing  high-speed  connec- 
tions between  CTC  and  MacCHESS.  the 
pioieci  IS  building  the  capability,  for  the  first 
time,  for  pharmaceutical  companies  and  aca- 
demic researchers  to  interact  dynamically  with 
x-ray  crystallographic  analyses  at  the  syn- 
chrotron, rather  than  discovering  long  after  the 
beam  run  that  the  sample  was  defective  or  the 
beam  positioning  non-optimal.  While  initially 
the  researchers  are  using  processors  on  the 
CTC  SP  systems,  ultimat'ely  a  small  IBM  SP 
may  be  installed  at  the  synchrotron  site  used 
for  dynamic  analysis,  with  longer-scale  simu- 
lation needs  being  met  using  the  far  larger 
SP2  at  the  CTC. 

PSC  is  working  with  the  Center  for  Light 
Microscopy  and  Biotechnology,  an  NSF  Sci- 
ence and  Technology  Center  at  Carnegie 
Mellon  University,  to  develop  an  Automated 
Interactive  Microscope.  This  microscope  will 
couple  leading  edge-microscopy  and  high  per- 
formance computing  through  high  speed  net- 
works allowing  the  real-time  tagging  of 
chemical  reactants  in  the  cell.  It  will  open  new 
research  horizons  in  biology  by  giving  re- 
searchers the  ability  to  control  the  release  of 
chemically  active  agents  at  critical  moments  in 
cell  life,  and  to  monitor  the  celFs  subsequent 
development. 

The  personal  computer  controlling  a  scanning 
tunneling  micro.scope  (STM)  in  the  Beckman 
Institute  at  UIUC  used  software  integrated 
over  a  LAN  with  the  NCSA  Convex  C3880. 
TMC  CM-5.  and  SGI  graphics  workstation  to 
enable  realtime  imaging  and  nanolithography 
of  silicon  surfaces  in  order  to  create  novel 
quantum  electronic  devices.  Working  with  the 
laboratory's  director,  Joseph  Lyding  (Dept.  of 
Electrical  and  Computing  Engineering,  UIUC 


and  Beckman  Institute)  and  iiuenioi  of  a 
widely  used  STM,  NCSA  staff  members 
Rachael  Brady  and  Clint  Potter  extended  this 
lelemicroscope  to  the  Internet  and  demon- 
strated the  feasibility  of  using  advanced  im- 
aging instrumentation  linked  with  advanced 
computing  capabilities  from  anywhere  in  the 
world.  The  project  was  featured  in  the  special 
issue  of  Research  and  Development  Mai^a- 
:ine  (Oct  25.  1993)  on  Winning  in  the  21st 
century. 

Nil  Testbeds 

As  a  partner  in  Common  Knowledge;  Pitts- 
burgh, an  innovative  project  introducing  net- 
working and  computing  into  the  entire  Pitts- 
burgh School  District.  PSC  is  working  with 
numerous  partners,  including  Digital  Equip- 
ment, Apple  Computer  and  both  telephone 
(Bell  of  Pennsylvania)  and  cable  TV  (TCI) 
companies,  to  create  a  prototypical,  cost- 
effective  approach  to  widespread  use  of  ad- 
vanced technology  in  public  education. 

In  collaboration  with  researchers  and  physi- 
cians at  the  University  of  Pittsburgh  Medical 
Center,  the  PSC  is  developing  an  Nil-based 
digital  library  of  pathology  images,  and  the 
applications  and  software  technology  which 
will  enhance  the  practice,  teaching  and  cost- 
effective  delivery  of  pathology. 

NCSA  in  collaboration  with  UIUC  and  the 
Champaign  County  Chamber  of  Commerce 
have  been  building  CCnet.  an  Nil  testbed 
during  the  last  18  months.  Over  200  people 
from  over  70  community  organizations  have 
been  involved  since  April,  1993  in  defining 
six  major  applications  experiments  in  small 
business,  health  care,  education,  government 
and  community  services,  agribusiness,  and 
geographic  information  systems  (GIS).  The 
first  of  20  multimegabil/s  links  into  the  com- 
munity was  established  with  the  Urbana  Free 
Library  in  August  1994.  All  the  high  .schools 
and  a  number  of  small  businesses  are  hooking 
on  in  September.  NCSA  is  establishing  a 
large  GIS  server  which  will  be  available  over 
CCnet  to  community  projects.  Partners  in 
CCnet        include        Time-Warner        cable. 
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Ameritech.  DEC,  Motorola,  and  potentially 
MCI  and  AT&T. 

NYNET,  one  of  the  gigabit  testbeds,  is  also 
designated  as  an  Nil  testbed  providing  out- 
reach, medical  appHcations,  video  on  demand 
to  CTC  and  New  York  academic  and  indus- 
trial partners. 

InterNIC  is  the  latest  in  an  evolutionary  line  of 
support  from  the  NSF  for  the  use  of  the  In- 
ternet by  the  science,  research,  and  education 
communities.  The  InterNIC  provides  three 
types  of  services:  Information  Services 
(provided  through  General  Atomics  and  the 
SDSC),  Directory  and  Database  Services,  and 
Registration  Services.  Information  Services 
provides  procedures  for  connecting  to  the 
Internet,  pointers  to  resources  and  tools  avail- 
able over  the  network,  training  seminars  for 
new  and  experienced  users  and  up-to-date 
reports  on  new  resources  and  activities  on  the 
Internet.  Several  innovative  approaches  to 
distributed  .services  have  been  implemented, 
including  the  InfoGuide,  an  on-line  Intemet 
information  service.  The  Scout  Report  is  a 
weekly  summary  of  Internet  highlights  which 
combines  in  one  place  the  highlights  of  new 
resource  announcements  and  other  news  that 
occurred  on  the  Internet  during  the  previous 
week.  The  InterNIC  Reference  Desk  acts  as 
the  "NIC  of  first  and  last  resort."  The  desk 
supports  a  variety  of  users  answering 
"starter"  questions  from  novice  users  who  are 
unfamiliar  with  the  Internet  as  well  as  spe- 
cialized questions  from  intermediate  and  ad- 
vanced users. 

Visualization  and  Virtual  Reality 

The  NSF  center's  were  instrumental  in 
bringing  the  notion  and  tools  of  scientific  vi- 
.sualization  to  the  research  community  in  the 
1980s.  By  combining  advanced  visualization 
resources  with  simulation  data.sets  created  by 
remote  users  on  the  centers  program's  high 
performance  computers,  new  visualization 
paradigms  for  interpreting  numerical  data 
were  developed.  This  led  scientists  to  consider 
visualization  as  an  intimate  part  of  their  com- 
putational toolkit.  In  addition,  the  centers 
worked  closely  with  the  pre-existing  computer 


graphics  community  to  get  them  creating  new 
tools  for  scientists  as  well  as  for  entertain- 
ment. Already  by  1 987.  the  staffs  of  the  cen- 
ters, working  with  national  users,  were  creat- 
ing scientific  visualizations  so  compelling  that 
they  became  regularly  chosen  to  be  part  of  the 
SIGGRAPH  Film  and  Video  Show,  the 
"academy  awards"  of  the  visualization  indus- 
try. Today  the  centers  visualization  staff  and 
their  allied  visualization  Centers  are  at  the 
forefront  of  research  into  how  to  turn  virtual 
reality  technologies  into  useful  tools  for  scien- 
tific and  engineering  research. 

Development  of  Scientific  Visual- 
ization 

From  its  inception,  NCSA  has  worked  with 
computer  artists  like  Donna  Cox  (UIUC  Dept. 
of  Art)  and  Dan  Sandin  (UIC  School  of  Art 
and  Design)  and  computer  scientists  like  Tom 
DeFanti  (Dept.  of  Electrical  Engineering  and 
Computer  Science,  UIC)  to  create  cross  disci- 
plinary teams  with  end  users  in  order  to  create 
new  levels  of  scientific  visualizations.  NCSA 
also  hired  a  number  of  staff  from  leading 
companies  in  the  California  entertainment 
industry  to  bring  the  software  tools  of  special 
effects  in  movies  or  TV  commercials  to  the 
use  of  the  scientific  and  engineering  commu- 
nities. Initially,  the  NCSA  visualization  envi- 
ronment was  built  on  Alliant  shared-memory 
multiprocessors  using  the  Wavefronl  visu- 
alization software.  As  specialized  graphics 
hardware  became  available  on  Silicon  Graph- 
ics systems,  the  NCSA  visualization  envi- 
ronment migrated  from  the  Alliant  to  Silicon 
Graphics  systems.  The  scientific  visualiza- 
tions created  by  NCSA  staff  have  not  only 
broken  new  ground  for  scientists  viewing 
their  data,  they  have  also  won  awards  world- 
wide for  aesthetic  quality.  NCSA's  Renais- 
sance Experimental  Laboratory  (REL),  cre- 
ated by  Donna  Cox  with  a  major  donation 
from  Jim  Clark  (founder  of  Silicon  Graphics) 
was  the  first  advanced  visualization  training 
facility  in  the  centers  and  continues  to  support 
university  courses  in  geology,  mathematics, 
graphics  design,  computer  .science,  and  other 
disciplines. 
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The  SDSC  has  developed  a  variety  of  soft- 
ware tools  that  can  be  used  to  access  and  con- 
nect existing  visualization  resources  automati- 
cally. The  goal  has  been  to  provide  researchers 
with  training  and  access  to  tools  that  will  sup- 
port their  research  needs.  The  tools  include:  1 ) 
The  SDSC  Image  Library,  a  collection  of  im- 
age manipulation  and  conversion  utility  rou- 
tines that  can  be  embedded  in  existing  soft- 
ware. An  interesting  example  of  the  use  of 
this  library  are  the  image  conversion  modules 
developed  by  the  International  AVS  Center, 
which  quickly  became  the  top  2  user  modules 
in  their  distribution;  2)  the  SDSC  Image 
Tools,  a  collection  of  utilities  based  on  the 
Image  Library,  are  software  tools  for  reading, 
writing,  and  manipulating  raster  images.  This 
toolset  also  allows  researchers  to  convert  file 
formats  among  over  thirty  widely  used 
graphics  formats  (e.g.,  from  HDF  to 
PICT). Now  in  its  .second  release,  it  runs  on 
Cray  Research.  DEC.  HP.  IBM.  SGI,  and 
Sun  Micro.systems  platforms.  Over  4,000 
sites  worldwide  have  uploaded  the  Image 
Tools  from  SDSC's  anonymous  FTP  area;  3) 
vpr,  a  client-server  visualization  hardcopy 
utility,  vpr  takes  advantage  of  the  Internet  by 
allowing  remote  users  to  send  images  to  local 
hardcopy  devices.  Popular  hardcopy  devices 
that  are  connected  to  vpr  include  a  variety  of 
film  recording  devices,  a  color  paper  plotter, 
and  video  recording;  and  4)  the  SDSC  Color 
Tutorial,  a  SuperCard-based  hypertext  explo- 
ration in  color  theory  for  computer  graphics. 
Examples  show  the  different  points  that  are 
being  made,  while  hyperlinks  allow  the  u.ser 
to  jump  to  the  most  interesting  references. 

CTC  Visualization  staff  developed  Visual 
Programming  Language  for  Animation 
(VPLA).  a  program  that  easily  integrates 
sound  and  image  .sequences  into  scientific 
animations.  VPLA  can  use  rendered  images 
from  several  standard  visualization  packages 
including  DataExplorer.  As  the  national  re- 
pository for  Data  Explorer  software  and  lead 
training  site,  the  CTC  works  with  faculty, 
industrial  users  and  students  across  the 
country  in  developing  state-of-the  art  anima- 
tions and  images.  In  addition,  using  DX,  the 


CTC  has  spearheaded  using  visual  program- 
ming languages  not  only  for  visualizing,  but 
for  managing  distributed  applications  running 
across  the  centers  program.  For  example,  the 
CTC  built  a  Data  Explorer  module  allowing 
its  researchers  to  access  the  CMS  at  NCSA. 
The  CTC  was  instrumental  in  IBM's  agree- 
ing to  support  DX  across  all  major  vendor 
platforms,  including  SGI. 

PSC  has  concentrated  its  visualization  efforts 
on  the  development  of  tools  for  remote  users. 
Its  GPLOT  software  is  in  use  at  over  300 
sites.  Its  automated  animation  facility  has  en- 
abled researchers  to  produce  hundreds  of 
videotapes  without  physically  visiting  the  any 
specific  center.  It  is  now  turning  its  efforts  to 
develop  such  tools  embodying  virtual  reality. 

Virtual  Reality  Impacts  Industrial  De- 
sign 

In  1992  NCSA  began  a  transition  from  the 
now-traditional  workstation  visualization  ac- 
tivities to  virtual  environments,  with  leader- 
ship provided  by  Caterpillar  Inc.,  an  NCSA 
Industrial  Partner.  Traditionally,  translating 
electronic  CAD  blueprints  into  full  scale 
wooden  models  of  new  heavy  earth  moving 
equipment  in  order  to  evaluate  design 
changes  required  6-9  months.  Working  with 
NCSA  staff.  Caterpillar  built  up  a  VR  labo- 
ratory in  the  UIUC  Beckman  Institute  and 
networked  the  SGI  graphics  workstations 
which  create  the  VR  images  to  their  Peoria 
headquarters  facilities.  Using  a  variety  of  VR 
viewing  technologies,  a  number  of  design 
options  already  have  been  tested  for  new 
models  of  Caterpillar  wheel  loaders  and 
backhoe  loaders  that  will  be  introduced  by 
1996.  Design  changes  can  now  be  made  in 
less  than  one  month.  Caterpillar  design  engi- 
neers Dave  Stevenson  and  John  Bettner  re- 
ceived the  1993  NCSA  Industrial  Grand 
Challenge  Award  for  their  innovative  work. 
Media  coverage  of  this  award  reached  over 
200  media  outlets. 
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Development  of  Immersive  Science 
Projects 

The  transition  from  stand-along  workstation 
visualization  to  Nil  distributed  visualization 
was  emphasized  at  SIGGRAPH"92  in  Chi- 
cago when  NCSA  collaborated  with  the  UI- 
Chicago's  Electronic  Visualization  Labora- 
tory and  dozens  of  science  teams  to  demon- 
strate wide  area  interactive  visualization  at 
Showcase.  A  new  level  of  realism  in  virtual 
reality  was  debuted  there  as  well  with  the 
public  showmg  of  EVL's  Cave  Automated 
Virtual  Environment  (CAVE),  which  pro- 
vided complete  immersion  in  complex  3-D 
data  sets  at  workstation  levels  of  resolution. 
In  1993-1994,  EVL,  NCSA,  and  Argonne 
organized  a  national  call  for  proposals  which 
resulted  in  over  60  EVL/NCSA  computer 
science  and  visualization  staff  and  graduate 
students  helping  researchers  from  over  30 
institutions  in  porting  their  applications  into 
the  CAVE  environment.  For  the  first  time, 
this  included  realtime  coupling  to  parallel  su- 
percomputers so  that  dynamic  3-D  evolu- 
tions could  be  viewed  immersively  and 
steered  interactively.  At  SIGGRAPH  94, 
8,000  attendees  were  able  to  directly  experi- 
ence these  science  projects.  CTC  developed  a 
specific  Cave  Visualization  on  Macromolecu- 
lar  Modeling:  the  Structure  of  Acetylcholine 
Esterase;  this  visualization  was  integral  to  the 
researcher's  understanding  of  the  molecule's 
activity.  This  application  runs  not  only  on  the 
CAVE'S  SGI  workstations,  but  on  CTC's 
IBM  SP2  system  as  well. 

Virtual  Reality  over  ATM  networks 

In  a  project  IN  1994  with  Rome  Laboratory, 
demonstrated  virtual  reality  techniques  over  an 
ATM  network  between  CTC  SGI  computers 
and  Rome  Laboratory.  Researching  ATM 
technologies  for  real  time  applications  and 
demonstrating  software  tools  for  application 
steering  important  to  molecular  modeling, 
telemedicine  and  command  and  control. 

The  Sequoia  2000  Visualization  Group  at 
SDSC  developed  a  prototype  data  visualiza- 
tion system  "Tecate"  using  virtual  reality 
technology  to  address  many  of  the  issues  in- 


volved in  exploring  the  informational  content 
of  networked  data  .servers.  Tecate  enables  the 
browsing  for  data  that  resides  in  repositories 
managed  by  a  database  management  system 
via  user-interaction  with  graphical  renditions 
of  objects  that  represent  data  features. 

Alpha  Shapes,  Biomolecules,  and 
Cosmology 

Alpha  shapes,  a  form  of  geometric  modeling 
developed  by  the  1993  Waterman  Award 
winner  Herbert  Edelsbruner  (Dept.  of  Com- 
puter Science,  UIUC)  and  NCSA  staff  mem- 
ber Ping  Fu,  focuses  on  the  formal  definition, 
construction,  and  measurement  of  shapes  for 
any  given  point  set  in  space.  The  discrete  na- 
ture of  the  alpha  shape  complex  has  computa- 
tional advantages  over  any  other  known 
method  which  can  be  exploited  in  computing 
surface  area  and  volume  of  a  space  filling  dia- 
gram and  in  localizing  and  measuring  voids. 
The  latter  is  useful  in  studying  water  mole- 
cules residing  inside  a  protein.  NCSA  users 
have  discovered  other  related  applications  of 
alpha  shapes  by  applying  them  to  such  diverse 
fields  as  adaptive  grid  generation,  medical 
image  analysis,  visualizing  the  structure  of 
earthquake  data,  and  the  large-scale  structure 
of  the  universe. 

Digital  Libraries  and  Infoservers 

The  National  Information  Infrastructure  re- 
quires many  software,  computer,  and  com- 
munications resources  that  were  not  tradi- 
tionally thought  to  be  part  of  high  perform- 
ance computing.  In  particular,  knowledge 
organization,  location,  and  navigating  tools 
needed  to  be  developed.  The  NSF  Su- 
percomputer Center  staffs  and  their  as- 
sociated universities  have  proven  to  be  fertile 
ground  for  developing  their  new  tools.  Per- 
haps the  most  spectacular  success  has  been 
NCSA  Mosaic,  which  in  less  than  18  months 
has  become  the  Internet  knowledge  browser 
of  choice  by  over  a  million  users.  The  Mo- 
saicAVorld  Wide  Web  infrastructure  has  set 
off  an  exponential  growth  in  the  number  of 
decentralized  authoring  of  information  serv- 
ers. 
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Digital  Libraries 

In  1989,  NCSA  as  part  of  its  XUNET  appli- 
cation testbed  proposed  that  multimedia 
digital  libraries  would  require  gigabit  net- 
works in  order  to  fully  support  high  defini- 
tion imagery  and  the  coupling  of  large  data 
sets  with  computing  resources  and  geo- 
graphically dispersed  researchers.  This  re- 
sulted in  research  developments  like  DICE 
(Distributed  Collaboration  Environment). 
Parallel  efforts  in  providing  researchers  with 
global  infonnation  retrieval  and  display  capa- 
bilities over  existing  environments  combined 
collaboration  with  Bruce  Schatz  (then  at  U. 
Arizona)  and  his  Worm  Community  Sys- 
tem, with  internet-based  designs  of  compo- 
nent client/server  architectures,  like  the  World 
Wide  Web.  These  approaches  influenced  the 
design  of  the  current  grand  challenge  digital 
library  prototype  for  accessing  radio  astron- 
omy images  and  data  sets.  NCSA  built  on 
the  success  of  Internet  access  tools  such  as 
NCSA  Telnet,  adapted  this  modified  digital 
library  paradigm  to  the  Internet  with  NCSA 
Mosaic.  Today,  the  grand  challenge  image 
library  uses  NCSA  Mosaic  as  it's  user  inter- 
face, and  Schatz  has  joined  NCSA  to  head 
the  recently  awarded  NSF/ARPA  Digital  Li- 
braries project,  which  combines  a  testbed 
based  on  the  component  architecture  with 
experiments  in  object-based  designs. 

Scalable  Information  Servers 

The  enormous  success  of  NCSA  Mosaic  and 
CERN's  WorldWideWeb  has  resulted  in 
explosive  growth  in  the  use  of  NCSA's 
WWW  server.  By  the  end  of  1993,  NCSA's 
server  load  had  grown  beyond  the  capabilities 
of  any  single  server.  This  resulted  in  the  de- 
sign of  an  innovative  distributed  scalable 
.server  architecture  that  involved  a  modifica- 
tion of  the  Internet's  Domain  Name  System 
software.  By  Sept.  1994,  the  NCSA  WWW 
server  was  handling  over  2  million  connec- 
tions per  week.  NCSA's  Hewlett-Packard 
workstation  cluster  based  distributed  infor- 
mation server  has  now  been  duplicated  at 
many  WWW  and  PTP  sites  on  the  Internet 
and  within  coiporations.  A  number  of  corpo- 


rations are  presently  working  with  NCSA  on 
the  next  generation  of  this  distributed  ar- 
chitecture. 

The  Rise  of  the  MosaicAVWW  In- 
formation Infrastructure 

NCSA  developed  the  Mosaic  user  interface 
software  which  provides  point-and-click  ac- 
cess to  the  diverse  information  storage  proto- 
cols of  the  Internet,  such  as  World  Wide  Web 
(WWW),  Gopher,  FTP,  and  WAIS.  NCSA 
Mosaic  establishes  the  necessary  connections, 
file  transmissions,  decompression,  launch  of 
viewer  programs,  and  screen  display  of  text, 
images,  animations,  or  audio,  in  response  to  a 
single  mouse  click  from  the  u.ser.  NCSA  Mo- 
saic is  available  for  Mac,  Windows,  and  Unix 
computers  for  free  to  individual  users,  for 
government  and  educational  use,  and  for  in- 
ternal use  within  companies.  Monthly  down- 
load rates  from  the  NCSA  site  alone  are  con- 
sistently over  30,000.  Although  accurate  esti- 
mates are  difficult,  it  is  widely  felt  that  over  a 
million  copies  of  NCSA  Mosaic  are  in  use. 
Further,  commercial  versions  of  NCSA  Mo- 
saic are  available.  The  principal  licensee.  Spy- 
glass, Inc.,  has  announced  orders  for  over  five 
million  copies  of  their  enhanced  version,  with 
projections  to  twenty  million  copies  within  a 
year.  Use  of  NCSA  Mosaic  has  increased 
WWW  traffic  on  the  NSF  backbone  by  over 
10,000  fold  since  Jan.  1993.  Overall  WWW 
traffic  in  August  hit  1.3  Terabytes  or  8  %  of 
the  total  NSFnet  backbone  traffic,  higher  than 
SMTP.  Because  of  this,  NCSA  has  become 
the  second  biggest  Internet  site  in  the  world  in 
terms  of  traffic  from  its  site.  The  NCSA  Mo- 
saicAVWW information  infrastructure  is  al- 
lowing for  an  enormous  growth  in  decentral- 
ized authoring  of  infoservers  throughout  the 
world.  In  1994,  NCSA  was  given  Infoworld's 
Publisher's  Industry  Achievement  Award. 

DESKTOP  Software 

From  the  beginning,  the  NSF  Supercomputer 
Centers  provided  focal  points  for  pulling  to- 
gether teams  of  computer  scientists  and 
software  developers.  Since  the  history  of  the 
centers  has  greatly  overlapped  with  the 
worldwide  rise  of  the  personal  computer  and 
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workstation,  it  is  not  surprising  that  the  soft- 
ware developers  focused  on  creating  easy-to- 
use  software  tools  for  the  desktop  machines 
themselves.  These  tools  have  had  a  major 
influence  on  the  usefulness  of  the  supercom- 
puter facilities  to  the  remote  science  and  en- 
gineering community.  The  collaboration  tools 
will  have  a  great  impact  on  tying  together  the 
newly  emerging  electronic  teams  of  .scientists 
made  possible  by  the  growth  of  the  Internet. 

Connectivity  Tools 

NCSA  Telnet  was  a  ground-breaking  desktop 
application  that  provided  access  to  the  emerg- 
ing NSF  Supercomputing  Centers  in  the  late 
■80s.  Developed  by  the  new  Workstation 
Tools  Group  (later  the  SDG)  at  NCSA,  this 
brought  full  TCP  connectivity  to  researchers 
using  IBM  and  Macintosh  sy.stems,  signifi- 
cantly broadening  the  participation  base  be- 
yond Unix  users,  thereby  introducing  thou- 
sands to  both  the  internet  and  the  NSF  Centers 
Program.  Continuously  supported  up  to  the 
present  time,  these  tools  have  also  led  to  a 
spin-off  company  Intercon,  headed  by  one  of 
NCSA  Telnet  developers. 

Collaboration  Tools 

NCSA  has  supported  a  program  of  research 
and  development  on  collaboration  technology 
for  science  and  engineering  researchers  for 
over  3  years.  NCSA  Collage,  a  tool  that  runs 
across  MSWindows,  Mac,  and  XWindows 
sy.stems,  provides  the  capability  to  carry  on 
remote  digital  conferencing  sessions  between 
researchers.  The  first  live  MetaCenter  collabo- 
rative session  using  NCSA  Collage  was  held 
in  1992  .  Collage  combines  many  of  the  fea- 
tures of  NCSA's  communications  and 
graphic  data  analysis  tools.  NCSA  also  con- 
tinues to  innovate  in  asynchronous  col- 
laboration tools  such  as  asynchronous  col- 
laboration tools,  hence  the  interest  in  annota- 
tion and  workgroup  support  capabilities  in 
NCSA  Mosaic.  Current  work  focuses  on 
choosing  and  combining  the  best  of  these  syn- 
chronous and  asynchronous  capabilities  in 
usable  next-generation  global  collaboration 
tools  for  the  scientific  and  educational  com- 
munities. 


Cornell  University's  CUSEEME  video  tele- 
conferencing .software,  aimed  at  providing 
video  teleconferencing  on  low-end  worksta- 
tions, is  in  use  at  the  NSF  Centers  and  as  part 
of  NYNET,  NYSERNET  and  other  organi- 
zations for  routine  use,  including  medical 
projects  between  the  CTC  and  the  Cornell 
University  Medical  College  in  NYC.  This 
work,  funded  by  NSF  and  Cornell  itself,  is 
freely  distributed  and  runs  on  Mac  and  PC 
platforms  using  inexpensive  video  equip- 
ment. Several  national  and  international  col- 
laborations have  successfully  utilized  this 
software.  The  centers  have  gained  experience 
with  traditional  video  teleconferencing  sys- 
tems, through  its  NSF-funded  system.  It  is 
now  looking  at  packet  video  systems  using 
the  vBNS  and  other  network  facilities.  These 
systems  will  have  the  capability  of  moving 
the  videoconference  from  a  set  of  specially 
equipped  rooms  to  the  desktop.  An  investi- 
gation is  now  underway  to  develop  the  opti- 
mal system  for  CTC's  requirements. 

Graphics  Tools 

NCSA  Image  was  the  first  scientific  vi- 
sualization tool  developed  for  the  desktop 
viewing  of  supercomputing  output  in  the  pro- 
gram. It  provided  the  research  community 
Mac  and  Unix  based  visualization  methods 
for  analysis  of  huge  data  sets,  as  well  as  cre- 
ating some  of  the  first  client/server  tools 
which  integrated  remote  desktop  workstations 
and  personal  computers  with  the  center  pro- 
grams high  performance  engines. 

The  SDSC  Image  Tools  are  software  tools  for 
reading,  writing,  and  manipulating  raster  im- 
ages. This  toolset  also  allows  researchers  to 
convert  file  formats  among  over  thirty  widely 
u.sed  graphics  formats  (e.g.,  from  HDF  to 
PICT)  and  includes  extensive  C  library  func- 
tionality for  creating  custom  image- 
manipulation  applications.  Now  in  its  second 
release,  it  runs  on  Cray  Research,  DEC,  HP, 
IBM,  SGI,  and  Sun  Microsy.stems  platforms. 
Binaries  and  sample  source  code  are  available 
in  the  public  doinain  by  accessing  SDSC's 
anonymous  ftp  area  (ftp.sdsc.edu). 
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Scientist's  Workbench 

The  Scientist's  Workbench  is  an  X  and  Motif- 
based  software  package  developed  at  the  CTC. 
The  main  functions  of  the  Scientist's  Work- 
bench are  to  bring  together  the  tools  and  soft- 
ware required  by  scientific  researchers  in  a 
distributed  computing  environment,  to  pro- 
vide a  graphical  interface  to  access  those  tools. 


and  to  provide  the  software  necessary  to  allow 
researchers  to  easily  build  their  own  graphical 
interfaces.  This  tool  has  been  used  at  most  of 
the  CTC's  Smart  Nodes  (affiliates)  by  users 
and  as  part  of  teaching  environments  for  high 
performance  computing,  as  well  as  at  the 
other  centers  and  by  companies  and  national 
labs  developing  "custom"  programming  in- 
terfaces for  their  communities. 


Accomplishments  in  Education  and  Outreach 


Education 

Familiarity  with  the  tools  of  computation  and 
visualization  is  quickly  becoming  a  sine  qua 
lion  for  both  researchers  and  the  public.  The 
spread  of  access  to  these  tools,  like  access  to 
the  telephone  and  television  before  them,  is  a 
democratizing  force  in  itself:  the  world  of  the 
shut-in  is  opened  up,  the  disadvantages  of 
distance  are  minimized,  the  exchange  of 
techniques  and  knowledge  is  enhanced.  Edu- 
cation, training,  and  outreach  are  thus  fun- 
damental to  the  programs  of  the  MetaCenter. 
Each  member  of  the  centers  program  has 
developed  educational  programs  targeted  to  a 
variety  of  constituencies:  university  research- 
ers, graduate  students,  undergraduates,  edu- 
cators at  all  levels,  and  K-12  students. 

Researchers  and  Students 

One-  or  two-day  workshops  are  offered  by 
centers  program  staff  to  researchers  on  site 
and  at  associated  institutions,  covering  intro- 
ductions to  the  computational  environments, 
scientific  visualization,  and  the  optimization 
and  parallelization  of  scientific  code.  In  ad- 
dition, special  workshops  have  been  offered 
throughout  the  centers  program  on  the  use 
and  extension  of  computational  and  visuali- 
zation techniques  specific  to  various  disci- 
plines (from  biochemistry  to  lattice  gauge 
theory).  On  the  campuses  of  centers  program 
institutions  and  on  other  campuses,  centers 
program  scientists  and  engineers  are  active 
teachers,  either  through  regular  academic  ap- 


pointments or  as  adjuncts,  lecturers,  seminar 
leaders,  or  teachers  in  extension  divisions. 

Graduate  students  often  receive  fellowships 
or  similar  appointments  at  centers  program 
institutions,  as  their  contributions  may  benefit 
a  large  academic  research  community  or  the 
computational  community  generally.  As  an 
example,  Ph.D.  student  Kimberly  Claffy 
(Computer  Science,  UCSD)  recently  com- 
pleted her  dissertation  on  a  flow-based  meas- 
ure of  Internet  traffic  that  she  developed  as  a 
Junior  Fellow  at  SDSC.  Dr.  Claffy's  tech- 
nique is  the  first  to  permit  traffic  characteri- 
zation on  the  basis  of  a  temporally  and  spa- 
tially flexible  unit,  and  it  is  thus  an  enabling 
technology  for  further  advanced  network  re- 
search at  SDSC  and  elsewhere. 

The  centers  program  has  contributed  to  the 
research  projects  of  hundreds  of  graduate 
students  through  stipends,  access  to  re- 
sources, and  relations  with  centers  program 
researchers.  Each  Center  fosters  collaborative 
research  by  multidisciplinary,  multi- 
institutional  teams  of  computer  scientists, 
research  scientists,  and  engineers;  postdoc- 
toral research  associates;  and  graduate  stu- 
dents from  the  national  and  international 
community.  These  teams  forge  new  ap- 
proaches to  previously  insoluble  research 
problems,  develop  community  codes,  and 
host  workshops  and  seminars  to  transfer 
technology. 
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Supercomputer  Centers  Educational  Activity  Support  Summary 


Educational  Activities 

FY91 

FY92 

FY93 

High  School 

Institutes 

7 

4 

5 

Attendees 

128 

131 

121 

Other  K- 12  Events 

15 

8 

17 

Attendees 

715 

1 .370 

1 .985 

Research  Institutes 

13 

1  1 

6 

Attendees 

262 

377 

390 

Training  CoursesAVorkshops                                                                              | 

On  Site  Events 

18 

102 

134 

Attendees 

1.414 

1.773 

1.929 

Off  Site  Events 

17 

23 

17 

Attendees 

295 

622 

104 

Seminars/Colloquia 

Events 

138 

114 

132 

Attendees 

2.251 

2.788 

3.085 

Academic  Course  Accounts 

64 

63 

79 

Monthly  Newsletter  Circulation 

234.986 

247.692 

165.176 

Visitors 

13.506 

16.380 

16.392 

For  undergraduates,  the  Research  Ex- 
periences for  Undergraduates  programs, 
funded  by  NSF,  bring  in  undergraduates  to 
work  for  a  summer  or  a  school  semester  or 
quarter  on  specific  projects  devised  by  cen- 
ters program  researchers  and/or  faculty  advi- 
sors. The  projects  are  significant  in  their 
scope  of  computational  science  and  in  many 
instances  have  resulted  in  presentations  at 
meetmgs  and  publications.  A  special  project 
is  CTC's  Supercomputing  Programs  for  Un- 
dergraduate Research  (SPUR),  in  which  stu- 
dents apply  to  work  on  one  of  a  selection  of 
projects  developed  by  Cornell  faculty  in  col- 
laboration with  CTC.  One  REU  student  at 
SDSC  went  on  to  win  the  top  prize  in  the 
Westinghouse  Science  Talent  Search  in  1991. 
Another  developed  a  program  to  teach  the 
use  of  the  Braillewriter  to  blind  students, 
which  was  presented  to  the  Commission  on 
Equal  Opportunity  in  Science  and  En- 
gineering at  NSF  (this  student,  herself  blind, 
is  now  a  successful  computer  scientist  in 
Silicon  Valley).  Undergraduate  assistantships 


and  internships  are  also  available  in  the  cen- 
ters program.  Undergraduate  student  pro- 
grammers have  worked  on  many  research 
problems  including  numerical  weather  pre- 
diction, the  visualization  of  numerical 
spacetimes,  and  social  network  analysis. 
They  have  developed  numerous  applications 
and  utilities  to  improve  the  computational  en- 
vironment for  MetaCenter  researchers.  Stu- 
dents have  also  worked  on  library,  visualiza- 
tion, and  educational  projects.  The  REU  pro- 
grams have  been  ongoing  in  various  forms 
for  more  than  five  years. 

Outreach  to  Educators 

One  particularly  effective  approach  to  edu- 
cating the  next  computational  generation  is 
the  training  of  teachers,  and  many  centers 
program  efforts  have  been  devoted  to  teacher 
training  and  curriculum  development. 

Common  Knowledge:  Pittsburgh  is  a  na- 
tional pilot  program  developed  by  PSC,  the 
University  of  Pittsburgh,  and  the  Pittsburgh 
Public  Schools  to  institutionalize  educational 
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technologies  within  the  Pittsburgh  PubHc 
School  District  by  having  PSC  implement 
the  network  infrastructure  and  develop  spe- 
cific curriculum-based  network  and  computer 
applications.  PSC's  High  School  Initiative 
(1992-1994)  involves  studentyteacher  teams 
using  PSC  facilities  to  develop  computational 
tools  for  inclusion  in  their  schools'  science  or 
mathematics  curriculum,  with  an  emphasis 
on  integrating  high-performance  computing 
into  the  curriculum  and  thus  bridging  the  gap 
between  textbook  instruction  and  real  world 
applications  of  science. 

SuperQuest  is  a  program  involving  centers 
program  sites  that  brings  teams  of  teachers 
and  students  from  selected  high  schools  to 
summer  institutes  to  develop  computational 
and  visualization  projects  that  they  work  on 
throughout  the  following  year.  In  addition  to 
educational  workshop  programs  associated 
with  SuperQuest.  NCSA  has  developed  five 
interactive  simulation  programs  now  being 
tested  in  classrooms  across  the  country  and 
around  the  world.  These  include  GalaxSee, 
an  N-body  simulator  of  galaxy  formation  and 
interaction;  the  Fractal  Microscope,  which 
enables  the  exploration  of  self-similar  pat- 
terns; SimSurface  and  SimElevator,  simu- 
lated annealing  programs;  and  LaplaceSeein', 
an  electrostatic  potential  solver.  Students  can 
change  initial  conditions  and  watch  the 
simulation  evolve  as  the  parameter  space  is 
explored. 

SDSC's  computational  .sciences  curriculum 
coordinator,  Kris  Stewart  (who  is  a  professor 
of  mathematics  at  San  Diego  State  Univer- 
sity) has  conducted  summer  workshops, 
funded  by  NSF  and  Cray  Research,  with  fac- 
ulty from  primarily  undergraduate  institu- 
tions to  develop  ways  of  incorporating  high- 
perfoiTnance  computing  into  the  curriculum. 
Stewart  uses  the  workshop  materials  in  her 
own  SDSU  classes,  Supercomputing  for  the 
Sciences  and  an  Introduction  to  Compu- 
tational Analysis.  SDSC  is  now  halfway 
through  a  three-year,  NSF-funded  Super- 
computer   Teacher    Enhancement    Program 


targeted  to  high-school  teachers  whose 
classes  contain  underrepresented  minorities. 

Dr.  Bruce  Land,  of  the  CTC,  has  developed 
an  undergraduate  course  in  scientific  vi- 
•sualization  and  computer  graphics,  using  the 
data  flow  block  diagram  capabilities  of 
IBM's  Data  Explorer  software.  This  cur- 
riculum, lab  exercises  and  the  resulting  stu- 
dent projects  have  been  shared  with  the  larger 
educational  community  through  the  CTC's 
Education  and  Training  home  page.  Addi- 
tionally, Prof.  Steve  Vavasis  has  developed 
an  interdisciplinary  course  in  .scientific  com- 
puting using  high  performance  computing 
for  graduate  and  undergraduate  students. 

Over  the  past  year,  the  CTC  had  established 
the  Data  Explorer  repository,  a  full  .set  of  tu- 
torials for  parallel  computing  on  diverse  plat- 
forms, a  complete  .set  of  lecture  notes  for  use 
by  educators  as  well  as  researchers,  and  a 
gateway  to  materials  on  the  network  for  sec- 
ondary school  .science  and  mathematics  edu- 
cation. 

The  educational  outreach  programs  of  the 
centers  program  enable  students  to  expe- 
rience the  advantages  of  connectivity  and 
training  in  all  aspects  of  modern  compu- 
tational practice.  The  challenge  to  effectively 
deliver  centers  program  resources  to  all  class- 
rooms is  being  met  mainly  by  the  distance- 
defeating  and  multiplicative  effects  of  high- 
performance  computation  itself.  The  dis- 
semination of  training  and  curriculum  mate- 
rials over  the  National  Information  Infra- 
structure is  a  major  way  in  which  the  suc- 
cessful pilot  programs  can  be  turned  into  a 
new  class  of  educational  resources. 

Outreach 

Because  advances  in  high-performance  com- 
puting and  communications  (HPCC)  are 
driven  by  the  needs  of  the  practitioners  with 
the  most  advanced  problems,  the  centers  pro- 
gram's .scientific  mission  includes  the  con- 
struction of  an  extensive  web  of  relationships 
with  research  and  development  efforts  in 
American  industry  and  commerce.  Collec- 
tively, the  centers  program's  outreach  pro- 
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grams  represent  a  long  record  of  sustained 
collaboration  among  scientists,  HPCC  de- 
velopers, and  industrial  researchers.  Another 
aspect  of  outreach  is  the  effort  to  find  and 
serve  local  and  regional  needs  of  govern- 
ment, schools,  and  communities.  Some  as- 
pects of  these  activities  are  discussed  below. 
Application  of  Scientific  Computation 
and  Visualization  to  Industrial  Pro- 
duction 
Half  of  the  partnerships  between  the  individ- 
ual centers  and  industry  are  collaborations 
with  major  industrial  firms.  These  include 
American  Cyanamid,  Amoco,  Alcoa, 
AT&T.  Caterpillar,  Corning,  Dow  Chemical, 
Eastman  Kodak,  Eli  Lilly,  FMC,  Gencorp. 
General  Dynamics.  Hughes  Aircraft,  IBM, 
JP  Morgan,  Martin-Marietta,  McDonnell- 
Douglas,"  Merck  Research  Labs.  Motorola. 
Parke-Davis.  Philips  Petroleum.  Schlumber- 
ger,  USX,  and  Xerox. 

In  their  original  form,  the  partnerships  repre- 
sented the  first  introduction  of   large-scale 
computation  and  visualization  into  the  store 
of  resources  possessed  by  even  the  largest  ot 
these    Fortune    500   companies.    While    the 
companies  are  for  the  most  part  fully  com- 
puterized now,  the  majority  of  these  partner- 
ships  continue  today   because  centers  pro- 
gram expertise  has  been  essential  to  the  in- 
troduction of  new  ways  of  employing  the 
resources  of  supercomputing:  the  algorithms, 
visualization  routines,  and  engineering  codes 
are  being  combined  in  ways  that  result  in 
such  advances  as  high-end  rapid  prototyping 
of  new  products.  As  a  result,  for  example. 
Eli  Lilly  maintains  its  partnership  although 
the  company  has  purchased  its  own  super- 
computer— the  useful  interactions  with  cen- 
ters   program    scientists,    consultants,    and 
visualizers  continue.  In  many   of  these  ar- 
rangements,    the     industrial     partner's     re- 
searchers are  frequent  visitors  to  the  NSF 
centers,  and  centers  program  researchers  also 
visit  the  partner's  installations. 

Thus,  while  it  is  extremely  important  to  Al- 
coa that  it  was  able  to  produce  an  optimum 
aluminum  can.  to  Gencorp  that  it  was  able  to 


design  a  better  injection  molding  process,  to 
McDonnell-Douglas  that  it  could  perform 
rapid  airfoil  analyses,  to  American  Cyanamid 
that  it  could  reformulate  soil  enhancers,  the 
sum  of  these  long-term  relationships  is  im- 
portant in  another  dimension  as  well.  The- 
Center  outreach  efforts  are  helping  to  revital- 
ize American  industry,  making  it  more  com- 
petitive in  an  increasingly  competitive  world 
market. 

Impact  on  Vendors  of  High  Per- 
formance Computing  Equipment 

The  centers  program  has  had  a  major  impact 
on  the  vendors  of  major  high  performance 
computing    equipment.    All    Centers    have 
taken  early   prototypes   of   machines,    have 
provided  national  access  to  the  largest  scale 
version  of     such   machines,   and  provided 
critical  feedback  to  the  vendors.  Several  have 
entered    into   strategic   development    efforts 
with  the  vendors.  For  example,   PSC    is  a 
formal  partner  with  Cray  Research  in  the  de- 
velopment of  applications  for  its  massively 
parallel  T3D.      Cray    has    also    internalized 
some  of  PSC's  file  sy.stem   developments. 
CTC  played  an  integral  role  in   IBM's   re- 
entry into  the  High  Performance  Computing 
arena,  as  the   first   customer   for   its   IBM 
ES/3090   vector  supercomputers  and   as   a 
partner  in  the  design  and  development  of  its 
parallel  FORTRAN  products.  CTC,  through 
Its   director  Malvin  H.  Kalos,  was  a  key  in- 
fluence in  IBM's  decision  in   1991   to  build 
the  IBM  SP  systems  and  ensured  that  IBM 
adopted  a  strategy   that   was    scalable   ulti- 
mately up  to  the  teraflops  and  down  to  the 
desktop. 

SDSC  has  established  a  clo.se  collaboration 
with  the  Supercomputer  Systems  Division 
(SSD)  of  Intel  Corporation  to  develop  sys- 
tems software  to  support  multi-user  systems, 
to  serve  as  a  test  site  for  new  operating  sys- 
tem releases,  and  to  improve  the  .stability  of 
the  Paragon  system.  SDSC  staff  have  de- 
veloped MACS,  the  Multi-user  Accounting 
and  Control  Sy.stem,  which  Intel  offers  as 
part  of  the  Paragon's  standard  operating  sys- 
tem software.  This  system    includes  a  dy- 
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namic  job  mix  scheduling  algorithm,  a  port 
of  the  Network  Queuing  System  batch  job 
submission  software,  and  CPU  quota  and 
accounting  systems  to  control  resources  used 
by  separate  projects. 

SDSC  has  also  collaborated  with  Cray  Re- 
search to  develop  support  for  multiple-user 
systems  on  Cray  systems.  SDSC  staff  have 
developed  a  resource  management  system 
that  controls  access  to  various  resources  on 
the  system  (CPU.  memory,  and  disk)  and  a 
dynamic  job  mix  scheduler  (DJMS)  to  dy- 
namically adjust  the  workload  for  optimal 
performance.  SDSC  has  recently  run  a  T3D 
emulator  on  the  Cray  C90  and  is  providing 
feedback  on  its  performance.  SDSC  now 
plans  to  install  and  evaluate  Cray's  new 
FDDI  card,  a  fiber-optic  high-speed  network 
interface. 

Digital  Equipment  Corporation  recently 
awarded  UCLA  and  SDSC  an  external  re- 
search grant  to  acquire  nine  Alpha  3000 
model  400  workstations.  The  cluster,  which 
has  a  peak  speed  of  1 .2  Gflops  and  is  con- 
nected at  100  Mbps  via  a  Gigaswitch,  will  be 
used  primarily  for  climate  studies  led  by  Dr. 
Roberto  Mechoso  of  UCLA.  It  will  also  be 
available  for  scientific  use  and  performance 
testing  by  the  SDSC  user  community.  SDSC 
staff  are  collaborating  with  DEC  to  port  the 
global  climate  model  to  the  Alpha  cluster 
using  DEC's  High  Performance  Fortran 
compiler. 

DEC  is  a  major  supporter  of  Project  Sequoia 
2000,  a  collaboration  of  scientists,  computer 
and  information  experts,  government  agen- 
cies, and  industrial  sponsors  to  develop  an  in- 
formation-management system  for  studying 
global  climate  change.  The  Sequoia  vi- 
sualization group,  centered  at  SDSC,  has 
been  developing  a  system  that  will  build  on 
the  strengths  of  existing  hardware  and  soft- 
ware to  support  next-generation  visualiza- 
tions. Recently,  the  group  collaborated  with 
Kubota-Pacific  Computer  Corporation  to 
combine  Kubota's  Denali  system  with  DEC 
Alpha  machines  for  advanced  3D  color 
graphics    capabilities.    Such    collaborations 


have  benefited  DEC.  Kubota-Pacific.  and  the 
Sequoia  2000  project. 

The  National  Storage  Laboratory  (NSL).  a 
consortium  that  is  developing  next-generation 
high-speed  storage  devices,  has  selected  the 
UniTree  system  as  the  production  archival 
storage  system  for  all  centers  program  sites. 
UniTree,  which  was  originally  developed  at 
the  Lawrence  Livermore  National  Labora- 
tory, was  commercialized  by  DISCOS,  a 
spin-off  of  SDSC  and  a  former  division  of 
SDSC's  parent  company.  General  Atomics. 
DISCOS  was.  in  turn,  sold  to  OpenVision. 
which  continues  to  market  the  product. 

SDSC  staff  are  collaborating  with  NSL. 
IBM.  and  OpenVision  to  implement  the 
NSL's  base  version  of  the  UniTree  archival 
storage  system  on  the  center's  IBM  RS/6000 
model  980  workstation.  They  are  adding  a 
new.  more  robust  name  server  developed  by 
Lawrence  Livermore  National  Laboratory  for 
OpenVision,  transaction  journaling  (which 
allows  reconstructing  the  database  in  case  of 
catastrophic  failure),  and  adding  and  enhanc- 
ing system  administrator  tools. 

The  integration  of  PSC's  Multi-resident  AFS 
into  the  NSL  UniTree  environment  is 
planned  to  provide  user-friendly  access  for 
the  centers  program  members  into  SDSC'S 
NSL  archival  storage  system.  The  future  in- 
tegration of  HIPPI-attached  peripherals,  in- 
cluding high-speed,  high-density  tape  and 
high-speed  RAID  disk  arrays,  using  third 
party  data  transfer  is  planned  as  a  way  to 
substantially  increase  both  archival  system 
storage  capacity  and  data  transfer  speed. 

A  prototype  HPSS  parallel  I/O  archival  stor- 
age system  is  also  planned  for  evaluation  as 
the  follow-on  to  the  NSL  UniTree  system. 
This  system  will  support  striping  across 
multiple  high-speed  peripherals  to  even  fur- 
ther increase  the  speed  of  file  transfers. 

Stimulation  of  New,  Computationally 
Dependent  Ventures 

About  a  fourth  of  the  industrial  partnerships 
are  with  smaller  and  newer  firms,  many  of 
them   leaders  in    biotechnology.    Some    are 
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firms  designing  new  pharmaceuticals  (e.g., 
Agouron  Pharmaceuticals,  Genentech),  oth- 
ers develop  and  market  the  software  pack- 
ages required  for  these  enterprises.  Biosym 
Technologies,  for  example,  is  working  with 
both  CTC  and  SDSC  to  develop  parallel  ver- 
sions of  its  popular  Discover  and  Insight 
packages. 

Outreach  efforts  of  the  centers  program  have 
resulted  in  actual  spinoff  ventures  as  well. 
The  commercialization  of  the  software  devel- 
oped at  individual  centers  is  being  undertaken 
by  a  number  of  companies.  For  example, 
NCSA  Telnet  has  been  commercialized  by 
Intercon,  and  Spyglass  will  relea.se  a  package 
containing  upgraded  versions  of  image  tools 
and  Mosaic.  Some  20  companies  have  now 
licen.sed  NCSA  Mosaic.  CERFnet,  a  Califor- 
nia wide-area  network  for  Internet  access  has 
pioneered  in  supplying  access  to  library  and 
other  large  databases;  and  DISCOSAJniTree, 
a  mass  storage  system,  is  in  use  at  more  than 
twenty  major  computer  sites.  A  new  mo- 
lecular modeling  system,  called  Sculpt,  de- 
veloped at  SDSC,  is  being  commercialized 
by  a  new  company.  Interactive  Simulations. 
Sculpt  enables  drag-and-drop  molecular 
modeling  in  real  time  while  preserving 
minimum-energy  constraints;  its  output  was 
featured  on  the  cover  of  Science  last  May. 

Development  of  Nationally  Valuable 
Reservoirs  of  Skill 

About  a  fourth  of  the  partnerships  between 
the  centers  and  industry  are  collaborations 
with  manufacturers  of  high-performance 
computational,  networking,  telecommunica- 
tions, and  visualization  equipment.  Of  par- 
ticular interest  here  are  the  several  partner- 
ships funded  by  the  NSF  and  ARPA  through 
CNRI  to  construct  and  test  the  "gigabit  test- 
beds,"  prototypes  of  the  connectivity  that  will 
be  required  for  the  future  International  In- 
formation Infrastructure.  Both  academic  and 


industrial  research  groups  are  developing  the 
codes  to  test  the  connections,  even  as  manu- 
facturers develop  the  connections  themselves, 
and  the  experts  assembled  in  the  centers  pro- 
gram supply  links  in  the  form  of  .specifica- 
tions and  software. 

Community  Service 

Local  and  regional  outreach  efforts  range 
from  the  tours  given  at  all  centers  program 
installations  through  the  hosting  of  visits  by 
national,  regional,  and  local  officials  and 
commissions,  to  the  kinds  of  full-scale  part- 
nerships mentioned  above.  The  NCSA  rela- 
tionship with  the  Champaign  County  Cham- 
ber of  Commerce  has  resulted  in  the  forma- 
tion of  a  nonprofit  public  network,  CCnet, 
which  is  already  benefiting  the  Chamber  it- 
self as  well  as  local  schools.  Plans  are  in  the 
works  with  Time-Warner  to  start  pilot  tests 
of  the  use  of  public-access  cable  for  a  data 
highway. 

SDSC  is  working  with  the  City  of  San  Diego 
on  plans  to  connect  all  units  of  city  gov- 
ernment, including  a  high-technology  re- 
source center  to  be  developed  with  De- 
partment of  Commerce  funding  that  will 
connect  local  industry  (with  a  lot  of  defense 
reconversion  efforts)  to  business  and  com- 
putational resources,  including  SDSC  itself. 

PSC  is  exploring  extension  of  the  technology 
it  developed  for  its  Common  Knowl- 
edge;Pittsburgh  K-12  project  to  embrace 
major  units  of  city  government. 

Outreach  is  also  represented  by  the  publica- 
tioi  programs  of  the  centers  program,  the 
production  of  scientific  videos  and/or  multi- 
media CD-ROMs,  and  a  collaborative  pro- 
gram for  maintaining  a  lively  and  informative 
presence  on  World-Wide  Web  servers, 
which  will  make  information  on  the  pro- 
grams    easily     accessible     over     the     NIL 


Important  Science  and  Engineering  Accomplishments 

Summaries  of  computationally  interesting  problems  in  the  NSF  Centers  Program  by  the 
national  science  and  engineering  communities: 
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Quantum  Physics  and  Materials 

The  great  disparity  between  nuclear,  atomic, 
or  molecular  scales  and  macroscopic  material 
scales,  implies  that  vast  computing  resources 
are  needed  to  attempt  to  predict  the  char- 
acteristics of  bulk  matter  from  fundamental 
laws  of  physics.  Therefore,  it  is  not  surpris- 
ing that  since  the  beginning  of  the  NSF  Cen- 
ters program  this  area  of  science  has  brought 
us  some  of  the  largest  users  of  supercom- 
puters. Materials  .scientists  have  often  been 
among  the  first  group  of  researchers  to  try 
out  new  architectures  that  promise  even 
higher  computational  speeds. 

Below  are  outlined  some  outstanding  exam- 
ples of  studymg  properties  of  bulk  matter 
from  extreme  conditions,  such  as  occur  in 
nuclear  collisions,  the  early  universe,  or  in  the 
core  of  Jupiter;  new  materials  such  as  nan- 
otubes  and  high  temperature  super- 
conductors; and  more  practical  materials  used 
today  such  as  magnetic  material  and  glass. 

Phase  Transition  in  QCD 

The  MIMD  Lattice  Calculations  Col- 
laboration (MILC)  is  attacking  the  Grand 
Challenge  problem  of  "the  origins  of  mass." 
Their  objective  is  to  use  the  theory  of  the 
forces  governing  what  are  called  the  "strong 
interactions"  of  elementary  particles  (quarks 
and  gluons)  to  calculate  the  observed  masses 
and  interactions  of  the  particles  that  are  made 
out  of  them:  the  hadrons,  which  include  the 
familiar  proton  and  neutron.  The  theory  is 
called  quantum  chromodynamics  (QCD), 
and  its  numerical  incarnation  is  called  "lattice 
gauge  theory,"  because  the  quarks  and  glu- 
ons are  represented  on  a  four-dimensional 
space-time  lattice.  They  have  published  nu- 
merous studies  of  the  mass  spectrum  of  the 
hadrons;  the  transition  between  ordinary 
matter  and  the  quark-gluon  plasma,  which  is 
important  in  the  study  of  the  conditions  of  the 
early  universe;  and  the  decays  of  hadrons  via 
weak  interactions.  A  number  of  investigators, 
coordinated  by  Robert  Sugar  (Dept.  of 
Physics,  UCSB),  are  engaged  in  this  project 
including:     Claude     Bernard     (Washington 


Univ.),  Thomas  A.  DeGrand  (Univ.  of  Colo- 
rado), Carleton  DeTar  (Univ.  of  Utah),  Ste- 
ven Gottlieb  and  Alexander  Krasnitz  (Indiana 
Univ.).  Douglas  Toussaint  (Univ.  of  Ari- 
zona). Julius  Kuti  (UCSD).  The  consortium 
has  used  large  allocations  of  time  on  a  wide 
range  of  MetaCenter  computational  facilities 
including:  Intel  Paragon  (SDSC),  TMC  CM- 
5  (NCSA),  clustered  IBM  RS/6000s  under 
PVM  (CTC.  NCSA),  Crav  Research  C-90 
(SDSC  and  PSC) 

Phase  Transitions  of  Solid  Hydrogen 

Calculations  by  Natalie,  Martin  and  Ceperley 
(Dept.  of  Physics  UIUC,  NCSA),  carried  out 
on  the  CRAY  Y-MP  at  NCSA  have  estab- 
lished the  series  of  crystalline  phase  transi- 
tions of  hydrogen  as  it  is  compressed  to  sev- 
eral million  atmospheres  of  pressure,  such  as 
found  in  the  interior  of  the  giant  planets. 
Since  Wigner  and  Huntington  in  1935 
pointed  out  that  a  transformation  from  a  mo- 
lecular to  atomic  state  is  inevitable  at  high 
pressure,  there  have  been  extensive  specula- 
tions on  when  and  how  this  transformation 
would  take  place.  The  recent  development  of 
the  diamond  anvil  technique  have  allowed 
experiments  to  be  performed  at  pressure 
slightly  lower  than  the  atomic  transition. 
Those  experiments  confirmed  the  existence 
of  an  molecular  orientation  transition  which 
had  been  earlier  computationally  predicted  by 
Ceperley  at  1.5  Mbar.  Extensive  and  highly 
accurate  quantum  Monte  Carlo  calculations 
on  a  variety  of  crystal  structures  now  predict 
that  the  metallic  transition  will  take  place 
from  a  distorted  molecular  hexagonal  struc- 
ture into  an  atomic  diamond  lattice.  For  this 
and  other  pioneering  science,  David  Ceperley 
was  awarded  the  fifth  Eugene  Feenberg 
Memorial  Silver  Medal  in  1994.  David's 
PhD  advi.sor  and  the  third  Feenberg  awardee 
is  Mai  Kalos.  Director  of  the  CTC.  himself  a 
major  user  of  several  MetaCenter  supercom- 
puters. 

Prediction  of  new  Nanomaterials 

Marvin  L.  Cohen  (NAS)  and  Steven  G. 
Louie  (Dept.  of  Physics,  UC  Berkeley)  have 
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used  MetaCenter  computational  resources 
(SDSC.  PSC,  NCSA  Cray  Research  Y-MP 
and  C90)  to  make  numerous  advances  in 
computational  materials  science.  Most  re- 
cently, they  have  used  both  first-pnnciples 
and  tight-bindmg  codes  to  examine  the  prop- 
erties of  carbon  nanotubes  and  nanotubes 
composed  of  boron,  carbon,  and  nitrogen. 
Carbon  nanotube.s-essentially  rolled  mi- 
crosheets  of  graphite-are  well  known,  thanks 
to  the  work  of  Sumio  lijima  and  colleagues  at 
NEC.  They  have  diameters  on  the  order  of  1- 
20  nm  and  are  producible  in  the  same  carbon 
arc  chambers  used  to  produce  fullerenes  (also 
called  "buckyballs"-  assemblages  of  60  or 
more  carbon  atoms  in  cage-like  structures). 
They  have  interesting  capillarity  and  elec- 
tronic properties.  Cohen.  Louie  and  their  col- 
leagues have  predicted  the  structure  and 
properties  of  nanotubes  made  of  boron  ni- 
tride, which  appear  to  be  more  stable  and 
controllable  in  terms  of  their  electronic  prop- 
erties. They  have  also  predicted  nanotubes  of 
BC_2N.  a  boron-carbon-nitrogen  compound, 
whose  electronic  properties  are  even  more  in- 
teresting: they  should  behave  like  nanoscale 
induction  coils.  Most  exciting,  the  structures 
that  were  predicted  computationally  are  now 
being  produced  experimentally  in  the  lab  of 
Alex  ZettI  at  Berkeley,  where  their  electronic 
properties  can  be  confirmed. 

Theory  of  High  Temperature  Su- 
perconductors 

The  Nobel  Prize  in  Physics  in  1987  was  for 
the  discovery  of  a  new  class  of  high  tem- 
perature superconductors.  Thousands  of  re- 
search papers  have  been  written  about  these 
unique  materials,  but  the  battle  is  .still  raging 
over  the  fundamental  mechanism  that  causes 
the  ^uperconducting  transition  at  Tc  ~  90K 
for  the  cuprate  oxides  such  as  YBA2CU3O7. 
David  Pines  (NAS  and  first  Feenberg 
Medalist)  and  Philippe  Monthoux 
(Department  of  Physics,  UIUC)  used  the 
NCSA  Cray  Research  Y-MP  to  carry  out  a 
strong  coupling  (Eliashberg)  calculation  of 
the  normal  state  properties  and  Tc  for  the 
model  experiment-based  magnetic  interaction 


between  quasiparticles.  They  found  that  when 
the  full  structure  of  the  quasiparticle  interac- 
tion is  taken  into  account,  a  superconducting 
transition  into  a  d-wave  planar  pairing  siaie 
occurs  at  Tc  ~  90K  for  comparatively  modest 
values  of  the  coupling  constant.  Although 
still  an  area  of  active  research,  this  computa- 
tion lends  credibility  to  the  model  that  it  is  the 
coupling  of  planar  quasiparticles  to  the  ex- 
perimentally measured  planar  electronic  spin 
fluctuation  excitations  which  determines  the 
normal  state  properties  (which  they  show 
acts  like  a  nearly  antifcrromagnetic  Fermi 
liquid)  and  makes  possible  high  temperature 
superconductivity. 

Magnetic  Materials 

James  Sethna  (Laboratory  of  Atomic  and 
Solid  State  Physics.  Cornell  Univ.)  uses  CTC 
parallel  supercomputers  to  study  the  dynam- 
ics of  disorder-driven  first-order  phase  trans- 
formations, including  3-D  numerical  simula- 
tions of  hysteresis  loops.  He  is  developing 
scalable  parallel  algorithms  for  systems  of 
size  N  =  2000^.  Sethna's  work  enables  the 
prediction  of  phase  transitions  with  critical 
tluctuations,  the  simulation  of  orders-of- 
magnitude  larger  systems  to  explore  critical 
phenomena,  and  detailed  computational 
studies  in  materials  science  as  applied  to 
magnetic  storage  media,  metallurgical  phase 
transformations,  and  gases  adsorbed  on  sur- 
faces. 

Ujiderstanding  Glass 

For  ab  initio  dynamical  calculations  to  be 
useful  for  real  materials  in  an  industrial  set- 
ting, they  must  be  able  to  deal  with  ensem- 
bles of  thou.sands  of  atoms  for  dynamical 
effects  modeled  over  microseconds.  Signifi- 
cant algorithmic  developments  made  jointly 
at  Corning.  Inc.  and  the  CTC,  coupled  with 
the  much  increased  capability  of  the  CTC's 
IBM  SP2  system,  allow  this  threshold  to  be 
crossed  for  the  first  time.  Postdoctoral  Fel- 
low Stefan  Goedecker,  hired  jointly  by 
Corning,  Inc.  and  CTC,  has  developed  new 
extremely  fast  ways  of  doing  tight-binding 
which  he  can  parameterize  with  the  ab  initio 
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codes.  This  is  the  only  approach  currently 
known  that  will  handle  thousands  of  atoms 
for  millions  of  time  steps,  bringing  the  re- 
searchers close  to  observing  many  of  the 
mysteries  involved  in  glass  chemistry  which 
have  been  not  well  understood  for  over  2000 
years. 

Biology  and  Medicine 

Living  creatures  exhibit  some  of  the  greatest 
complexity  found  in  nature.  Therefore,  su- 
percomputers have  made  possible  unprece- 
dented opportunities  to  explore  these  com- 
plexities ba.sed  on  the  fundamental  advances 
made  in  biological  research  of  the  last  fifty 
years.  These  activities  include:  inverting  the 
data  from  x-ray  crystallography  experiments 
to  obtain  the  molecular  structure  of  macro- 
molecules:  learning  how  to  use  artificial  in- 
telligence to  fold  polypeptide  chains,  deter- 
mined from  genetic  sequencing,  into  the 
three-dimensional  proteins;  and  determining 
the  function  of  proteins  by  studying  their  dy- 
namic properties,  as  well  as  how  they  interact 
with  each  other  or  with  the  DNA  backbone 
from  whence  they  were  created. 

These  insights  are  beginning  to  make  signifi- 
cant impacts  on  medicine  and  plant  and  ani- 
mal biology.  New  fields  of  computational 
science,  such  as  molecular  neurosciences,  are 
being  enabled  by  academic  access  to  Meta- 
Center  computing  and  visualization  resources 
and  staff.  Corporations  are  using  supercom- 
puters and  advanced  visualization  techniques 
in  collaboration  with  the  NSF  MetaCenter  to 
create  new  drugs  to  fight  human  disea.ses 
such  as  asthma.  New  insights  into  economi- 
cally valuable  bioproducts  are  being  gained, 
for  instance,  by  combining  molecular  and 
medical  imaging  techniques  to  create  "virtual 
spiders'"  which  can  be  digitally  dissected  to 
understand  the  production  of  silk.  Finally, 
high  performance  computers  are  just  be- 
coming powerful  enough  that  some  dedicated 
researchers  are  able  to  program  mathematical 
models  of  realistic  organ  dynamics,  such  as 
the  human  heart. 


Crystallography 

Herbert  Hauptman  (Medical  Foundation  of 
Buffalo.  Inc.)  won  a  Nobel  Prize  in  1985  for 
development  of  the  "direct  method"  of  pro- 
tein structure  determination  from  X-ray 
crystallographic  data.  In  a  collaboration  with 
Russ  Miller  (State  University  of  New  York 
at  Buffalo),  these  researchers  have  developed 
a  numerical  approach  that  extends  the  "direct 
method"  of  determining  molecular  structure 
from  X-ray  crystallographic  data  to  larger 
molecules,  beyond  its  present  limit  of  about 
100  atoms.  The  algorithm  they  have  devel- 
oped, called  "Shake-and-Bake,"  runs  on  a 
number  of  computing  platforms  (PSC  CM-2 
and  Cray  T3D,  NCSA  and  PSC  CM-5)  and 
has  proven  it.self  effective  in  more  than  20 
cases  at  accurately  determining  the  structure 
of  proteins  that  have  taken  as  long  as  10 
years  by  existing  methods,  reducing  the  time 
to  a  matter  of  hours. 

Folding  Proteins  using  Artificial  In- 
telligence 

One  of  the  most  pressing  problems  in  mo- 
lecular biology  is  how  to  determine  the  fold- 
ing and  3-D  structure  of  a  protein,  given  its 
sequence.  Peter  Wolynes  (NAS),  Zan 
Schulten.  and  coworkers  (Dept.  of  Chemis- 
try. UIUC)  have  developed  a  novel  approach 
to  this  classic  problem  using  elements  from 
the  theory  of  spin  glasses,  associative  mem- 
ory models,  and  neural  networks.  Spin  glass 
theory  provides  a  framework  for  under- 
standing the  cooperative  nature  of  the  folding 
transition  and  the  qualitative  nature  of  the 
phase  diagram  describing  the  thermo- 
dynamics of  proteins.  Wolynes  et  al.  devel- 
oped simulation  codes,  based  on  associative 
memory  Hamiltonians,  and  characterized 
their  phase  diagrams  .semi-quantitatively. 
These  Hamiltonians  are  ba.sed  on  an  energy 
function  which  correlates  the  sequence  of  the 
protein  to  be  folded  with  those  of  proteins 
whose  structure  is  known.  They  were  intro- 
duced several  years  ago  by  Wolynes  and  co- 
workers as  polymer  analogues  of  the  Hop- 
field  neural  nets.  Their  work,  carried  out  on 
NCSA's  Cray-2  and  CRAY  Y-MP,  shows 
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that  even  primitive  associative  memory 
Hamiitonians  can  recognize  protein  structures 
from  sequences  that  are  only  moderately  re- 
lated to  those  already  existing  in  the  database. 
These  procedures  are  somewhat  similar  in 
effectiveness  to  the  rule-based  homology 
modeling. 

Protein  Kinase  solution 

The  Computational  Center  for  Macro- 
molecular  Structure  (CCMS).  founded  in 
1990,  is  an  NSF-funded  joint  project  of 
UCSD.  SDSC.  and  The  Scripps  Research 
Institute,  with  collaborators  from  ail  over  the 
country.  The  center  made  headlines  in  1991 
when  a  group  led  by  Susan  Taylor  (Dept.  of 
Chemistry.  UCSB),  one  of  the  principal  in- 
vestigators of  CCMS  published  the  three- 
dimensional  structure  of  the  catalytic  unit  of 
cyclic-AMP-dependent  protein  kinase,  or 
cAPK.  This  was  the  first  kinase  structure  to 
be  solved.  The  solution  was  achieved  by  a 
combination  of  computational  methods,  in- 
cluding refinement  on  the  SDSC  CRAY  Y- 
MP  using  the  program  XPLOR,  developed 
by  Axel  Bruenger  of  Yale  University.  Most 
important  to  the  solution  was  the  ability  of 
the  scientists  to  study  .stereo  visualizations  of 
the  structure  on  a  large  screen  at  SDSC,  so 
scientists  from  every  discipline  within  the 
group  could  contribute  their  insight  to  a  col- 
lective determination  of  the  structure.  Kinases 
play  important  messenger  roles  in  cell  me- 
tabolism, and  hundreds  of  such  compounds 
have  been  identified  and  sequenced.  Becau.se 
sequences  are  homologous  in  long  stretches, 
the  solution  for  cAPK  is  proving  extremely 
valuable  as  a  template  for  modeling  and  de- 
riving the  structure  of  other  kinases.  Taylor 
and  her  group  have  collaborated  with  several 
other  groups  since  in  modeling  proposed 
solutions  for  other  kinases,  including  those 
known  to  have  carcinogenic  properties  or  to 
be  involved  in  other  disease  processes.  In  all 
of  these  studies,  computation  and  visualiza- 
tion have  played  an  important  role.  Solutions 
for  various  kinases  can  lead  to  the  design  of 
inhibitors  to  prevent  the  enzymes  from  acting 
to  produce  diseases.  The  work  won  the  Fore- 


fronts of  Large-Scale  Computation  Award 
presented  at  Supercomputing  '93. 

Molecular  Neuroscience-Serotonin 

A  number  of  cardiovascular  and  psychiatric 
diseases  are  currently  treated  with  drugs  that 
act  on  the  neurotransmitter  serotonin  and  its 
receptors.  The  cellular  receptor  for  serotonin 
is  a  gatekeeper  molecule  that  recognizes  and 
binds  the  serotonin  and  then  transmits  the 
signal  to  the  cell  by  binding  to  a  special  class 
of  transducers:  the  G-proteins.  Using  the 
CTC's  ES/9000,  Dr.  Harel  Weinstein, 
chairman  of  the  Department  of  Physiology 
and  Biophysics  at  Mount  Sinai  Medical  Cen- 
ter, made  a  breakthrough  in  modeling  the  se- 
rotonin receptor.  His  breakthrough  came 
from  modeling  the  structural  changes  that 
occur  in  the  serotonin  receptor  when  it  binds 
to  a  ligand  and  to  the  G-protein.  causing  it  to 
carry  out  its  function.  This  research  shows 
how  G-proteins  can  be  switched  on  by 
structural  changes  in  specific  regions  of  the 
receptor  molecule  and  is  expected  to  lead  to 
the  development  of  more  effective  drugs, 
specific  ligands  aimed  at  the  regions  where 
the  structural  change  takes  place.  Weinstein 
believes  that  his  work  may  be  applied  more 
broadly  to  other  receptor  molecules,  includ- 
ing all  neurotransmitters,  that  communicate 
with  cells  via  G-proteins.  If  Weinstein  can 
demonstrate  a  common  mechanism  of  re- 
spon.se  in  these  receptors,  he  will  have  a  new 
type  of  molecular  approach  to  treating  a  vast 
range  of  diseases.  Weinstein  has  also  u.sed 
the  Cray  Research  C-90  and  Intel  Paragon  at 
SDSC. 

Molecular  Neuroscience-Acetyl- 
cholinesterase 

A  collaboration  between  Michael  Gilson, 
T.P.  Straatsma,  and  Andrew  McCammon 
(Dept.  of  Chemistry.  University  of  Houston), 
Daniel  Ripoll  (Research  Associate.  CTC), 
Carlos  Faerman  (Dept.  of  Molecular  and  Cell 
Biology,  Cornell  Univ.).  Paul  Axelsen  (Dept. 
of  Pharmacology,  University  of  Pennsylva- 
nia School  of  Medicine),  and  Israel  Silman 
and  Joel  Sussman  (Weizmann  Institute  of 
Science,  Israel)  has  used  molecular  dynamics 
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algorithms  to  investigate  the  rapid  activity  of 
the  enzyme  acetylchoHnesterase  (AChE).  The 
enzyme  breaks  down  the  neurotransmitter 
acetylcholine  diffused  across  nerve  cell  syn- 
aptic gaps.  Its  three-dimensional  crystal 
structure  was  solved  by  Joel  Sussman  and 
colleagues  at  the  Weizmann  Institute  in  Re- 
hovot,  Israel,  several  years  ago.  That  struc- 
ture showed  the  active  site  to  be  a  long,  nar- 
row channel — too  narrow  to  deal  rapidly  with 
the  job  of  dissociating  acetylcholine  into 
choline  and  an  acetate  ion.  Yet  it  is  known 
that  AChE  acts  very  rapidly,  no  doubt  be- 
cau.se  of  aeons  of  evolutionary  pressure  to 
optimize  the  response  of  the  nervous  system 
in  all  organisms.  Molecular  dynamics  calcu- 
lations performed  on  the  MetaCenter's  Cray 
Research  C-90  (SDSC  and  PSC),  TMC  CM- 
5  (NCSA),  and  Kendall  Square  KSR-1 
(CTC)  showed  that  there  was  also  a  "back 
door"  to  the  active  site,  that  might  open  to 
facilitate  the  exit  of  the  acetate  ion  from  the 
site.  A  study  of  the  electromagnetic  fields  of 
acetylcholinestera.se  with  the  back  door 
closed  and  then  open  supplied  confirming 
evidence.  Since  inhibitors  of  AChE  are  im- 
portant medications  for  myasthenia  gravis, 
glaucoma,  and  Alzheimers's  disease,  this 
new  insight  may  lead  to  more  effective 
pharmaceutical  agents  to  fight  these  diseases. 
This  work  was  the  cover  story  of  the  March 
4,  1994  issue  of  Science  magazine. 

Kinking  DNA 

John  M.  Rosenberg  (University  of  Pitts- 
burgh) used  the  PSC  Cray  Research  C-90 
vector  supercomputer  to  determine  how  a 
protein  identifies  and  interacts  with  specific 
sites  of  DNA  —  a  fundamental  biological 
process  called  "protein-DNA  recognition," 
which  is  related  to  many  disease  processes 
and  is  also  a  vital  tool  in  the  biotechnology 
industry.  Rosenberg's  molecular  dynamics 
simulations  have  refined  the  structure  of  an 
important  protein,  Eco  Rl  endonuclease, 
u.sed  in  DNA  cloning,  and  they  have  resulted 
in  a  clear  understanding  of  a  "kink"  in  the 
DNA  backbone  that  results  when  Eco  Rl 
endonuclease  binds  with  DNA.    Rosenberg 


won  the  1991  Forefronts  of  Large-Scale 
Computation  award  for  this  research,  and  his 
work  was  cited  in  the  1993  Compute rwnrld 
Smithsonian  award  for  .science  given  to  the 
PSC. 

Antibody-Antigen  Docking 

A  collaboration  among  computer  scientists 
Michael  Hoist  and  Faisal  Saied  (Dept.  of 
Computer  Science,  UIUC)  and  two  biolo- 
gists Richard  Kozack  and  Shankar  Subra- 
maniam  (Dept.  of  Physiology  and  Biophys- 
ics. UIUC  and  Beckman  Institute/NCSA) 
has  been  able  to  solve  for  the  first  time  the 
complete  nonlinear  Poisson-Boltzmann 
equation,  which  is  the  fundamental  equation 
of  macromolecular  electrostatics.  A  method 
based  on  multigrid-inexact  Newton  algo- 
rithms has  been  developed  and  large  memory 
applications  run  in  parallel  on  the  NCSA 
Convex  C3  show  that  this  has  profound  con- 
sequences for  protein  structure,  enzyme 
mechanisms  and  protein  design.  Coupling 
this  new  approach  with  a  Brownian  dynam- 
ics method,  the  largest  simulation  ever  of  an 
encounter  between  two  proteins,  an  antibody 
and  an  antigen,  has  been  carried  out  using  the 
NCSA  CM-5  and  SGI  Challenge.  This 
simulation  for  the  first  time  is  able  to  give 
rate  constants  for  association  of  proteins  that 
is  comparable  to  experimental  measure- 
ments. The  results  of  the  electrostatics  work 
was  the  cover  story  of  the  March  1994  issue 
of  Proteins:  Structure,  Function,  and  Ge- 
netics. 

Tuning  Biomolecules  to  Fight  Asthma 

Over  the  last  20  years,  the  number  of  asthma 
cases  has  almost  tripled  in  the  U.S.  David 
Herron,  senior  research  scientist  at  Eli  Lilly 
and  Company,  is  searching  for  new  drugs 
that  will  inhibit  the  action  of  leukotrienes,  in- 
flammatory agents  released  by  several  types 
of  cells  in  the  lungs,  which  cause  the  lungs  to 
stiffen  and  become  imtated.  Several  gi- 
gabytes of  data  from  molecular  dynamics  of 
three  key  leukotrienes,  run  on  NCSA's  and 
Lilly's  Cray-2  supercomputer  were  analyzed 
in  a  lengthy  scientific  visualization  created 
with  NCSA  staff.  Using  the  animations  as  a 
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guide.  Lilly  asthma  researchers  synthesized 
highly  active  antagonists  against  the  leuko- 
trienes.  Some  have  been  tested  in  asthma 
sufferers  and  found  to  be  effective  medically. 
For  this  work,  Herron  was  a  co-recipient  of 
the  first  NCSA  Industrial  Grand  Challenge 
Award. 

Virtual  Spider  and  Artificial  Silk 

Biophysicist  Lynn  Jelinkski,  director  of 
Cornell  University's  Biotechnology  Center 
for  Advanced  Technology  (CAT)  is  com- 
bining medical  imagmg  techniques  with  the 
state-of-the-art  computer  visualization  re- 
sources of  the  CTC  to  study  the  molecular 
structure  of  the  strongest  silk  of  the  golden 
orb  weaver  spider  and  its  transformation 
from  a  viscous  fluid  into  the  extremely 
strong  cry.stalline  fiber  which  has  the  poten- 
tial to  replace  manmade  fibers,  such  as  nylon, 
manufactured  from  petrochemicals.  Jelinski 
has  devised  a  way  to  create  a  3-D  "computer 
spider"  by  compiling  stacks  of  the  2-D  MRI 
images  using  the  IBM  POWER  Visualiza- 
tion System,  one  of  the  high-performance 
computing  resources  of  the  CTC.  Each  im- 
age contains  over  100,000  pixels.  Hundreds 
of  images  are  combined  to  construct  the  3-D 
simulated  .spider.  Once  in  hand,  this  virtual 
spider  can  be  dissected  by  computer  to  de- 
scribe the  anatomy  of  the  glandular  system 
and  to  provide  the  physical  processing  in- 
formation Jelinski  seeks.  This  understand- 
ing, coupled  with  molecular-level  studies  of 
the  amirio  acids  that  make  up  the  web  silk 
polymer,  may  aid  in  genetically  engineering 
plants  to  produce  fibers  as  strong  as  those 
produced  by  the  spider.  Jelinski's  work 
blazes  a  path  toward  the  development  of  a 
new  class  textiles  with  superior  strength  at 
the  same  time  that  it  promises  fundamental 
insight  into  the  mystery  of  the  spider's  web. 

Heart  Modeling 

Charles  S.  Peskin  and  David  M.  McQueen 
(Courant  Institute,  New  York  University) 
have  developed  over  the  last  decade  a  fully 
functionmg  three-dimensional  model  of  the 
heart,  its  valves  and  nearby  major  vessels. 
This  computational  model  will  make  it  pos- 


sible to  study  questions  about  normal  and 
diseased  heart  function  that  are  difficult  or 
impossible  to  address  through  animal  and 
clinical  studies.  The  complexity  of  the  heart 
model  is  so  great  that  a  single  heartbeat  re- 
quires a  150  hour  run  on  the  PSC  Cray  Re- 
search C90  and  could  not  have  been  run 
without  the  very  large  memory  of  the  C90. 
This  research  won  the  1994  Computerworld 
Smithsonian  award  for  Breakthrough  Com- 
putational Science.  Peskin  was  awarded  a 
Mac  Arthur  Prize  Fellowship  in  1983. 

Engineering 

Man-made  devices  have  become  so  complex 
that  researchers  in  both  academia  and  indus- 
try have  turned  to  supercomputers  in  order  to 
be  able  to  analyze  and  modify  accurate  mod- 
els in  ways  which  complement  the  traditional 
experimental  methods.  Such  easily  accessible 
high  performance  computers  enable  aca- 
demic engineers  to  study  the  brittleness  of 
new  types  of  steel,  to  improve  bone  trans- 
plants, or  to  reduce  drag  of  flows  over  sur- 
faces using  riblets.  Industrial  partners  of  the 
individual  supercomputer  centers  within  the 
MetaCenter  are  using  computational  facilities 
more  advanced  than  they  have  access  to  in- 
temally  to  improve  indu.strial  proces.ses  such 
as  in  metal  forming.  Better  consumer  prod- 
ucts such  as  leakproof  diapers,  or  more  effi- 
cient airplanes  are  being  designed.  Even 
State  agencies  are  able  to  use  the  MetaCenter 
facilities  to  improve  traffic  safety  or  find  bet- 
ter ways  to  use  recycled  materials.  Some  70 
corporations  have  taken  advantage  of  the 
MetaCenter  industrial  programs  to  improve 
their  competitiveness. 

Ultra-high-strength  Steels 

Gregory  B.  Olson  and  Arthur  J.  Freeman 
(Northwestern  University)  use  computer 
modeling  to  design  ultra-high  strength  steel 
for  weight-critical  applications  such  as  naval 
aircraft  landing  gear,  high-performance  race 
cars,  and  bearings  in  the  main  engine  turbo 
pumps  of  the  space  shuttle.  In  recent  super- 
computer modeling  on  the  PSC  Cray  C90, 
applying  quantum  mechanical  calculations  to 
the  structure  of  steel,  they  have  explained  the 
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molecular  mechanisms  that  give  rise  to  im- 
purity-mduced  embrittlement  in  steel,  work 
which  IS  expected  to  lead  to  steel  that  will  not 
shatter  in  frigid  conditions.  This  work  was 
reported  in  the  July  15,  1994  Science.  Free- 
man and  his  group  have  been  users  of  NSF 
supercomputers  since  the  founding  of  the 
program  on  a  wide  range  of  problems  in 
materials  sciences.  In  recognition  of  his  pi- 
oneering work  in  computational  materials 
research.  Freeman  received  the  first  Materials 
Research  Society  Medal  and  the  first  Award 
in  Magnetism  from  the  lUPAP. 

Continuous  Casting  of  Steel 

Achilles  Vassilicos  (U.S.  Steel  Technical 
Center)  models  the  flow  of  molten  steel  in  a 
continuous-casting  "tundish"  on  the  PSC 
Cray  Research  C90,  resulting  in  improved 
process  control  over  the  quality  of  steel.  By 
more  accurately  predicting  the  precise  metal- 
lurgical composition  of  the  continuous- 
casting  output  "slabs,"  U.S.  Steel  reduces 
waste  steel  and  the  amount  of  inventory  it 
must  keep  on-hand,  resulting  in  substantial 
cost  savings. 

Beverage  can  design 

Three-dimensional  stress  modeling  of  alumi- 
num beverage  cans  on  the  PSC  C90  by  Rob- 
ert E.  Dick  and  Andrew  B.  Trageser 
(ALCOA  Laboratories)  has  greatly  reduced 
the  expense  of  developing  a  new  can  design 
that  will  meet  customer  specifications  for 
strength  and  appearance.  By  relying  less  on 
costly,  time-consuming  prototype  testing, 
ALCOA  engineers  estimate  a  co.st  savings  of 
SI 00,000  or  more  per  can  design.  This  re- 
search has  been  described  in  articles  in  D/,?- 
crnrr  (March  1991),  Business  Week  (Oct.  8, 
1990)  and  in  Science  (June  23.  1989). 

Designing  a  Leakproof  Diaper 

Designing  effective  and  comfortable  dispos- 
able infant  diapers  requires  greater  un- 
derstanding of  the  function  of  the  diaper 
components,  such  as  the  cellulosic  fluff  and 
the  superabsorbent  polymer  particles-and  the 
effect  of  variations  of  parameters  related  to 
these  components.  Dow  Chemical  Company. 


has  done  extensive  experimental  testing, 
evaluation,  and  computer  modeling  that  has 
contributed  to  a  faster  developmental  process 
and  shortening  the  time  for  new  product  in- 
troduction. The  innovative  Dow  design  was 
evaluated  using  a  computer  model  run  on 
NCSA's  CRAY-2  system.  Three  separate 
time-dependent  processes  are  modeled.  The 
first,  a  fast  spreading  process,  involves  the 
insult  on  the  pad  by  a  quantity  of  liquid 
(urine)  which  is  transported  through  the  pad 
by  wicking.  During  this  process  the  fluff  pad 
collapses  as  it  becomes  wet.  The  second, 
called  the  imbibition  process,  models  the 
swelling  of  the  superabsorbent  polymer  par- 
ticles and  their  uptake  of  liquid  from  the  cel- 
lulose fluff.  During  this  process,  the  fluff  ex- 
pands again.  The  slowest-and  final-process 
tracks  the  redistribution  of  liquid  in  the  fluff 
pad  as  the  saturation  of  the  fluff  adjacent  to 
the  superabsorbents  changes.  The  overall 
model  was  compared  to  a  magnetic  reso- 
nance imaging  experiment,  which  provides  a 
three-dimensional  image  of  the  water  distri- 
bution in  a  diaper,  and  was  shown  to  give 
comparable  results  to  the  final  steady-state 
values.  Optimization  of  these  processes  is 
leading  to  an  improved,  quality  diaper. 

Bone  Transplant  Bioengineering 

Dean  Taylor  and  Donald  Bartel  (Dept.  of 
Mechanical  and  Aerospace  Engineering, 
Cornell  University)  have  been  able  to  investi- 
gate bone-implant  systems  across  a  wide 
range  of  design  parameters  by  using  high 
performance  parallel  computing  (including 
the  IBM  SPI)  and  visualization  resources  at 
CTC.  Their  long-term  research  has  produced 
models  of  the  stresses  placed  on  normal 
bones  and  the  artificial  components  of  a  hip 
joint — these  models  have  led  to  custom- 
designed  prostheses  and  reduced  prosthesis 
replacement  surgery. 

Improving  Performance  with  Riblets 

George  Em  Karniadakis'  and  his  group  at 
Brown  University  are  using  the  SDSC  Intel 


Uncidentally,  Prof.  Karniadakis  is  the  current 
chair  of  the  joint  NCSA /PSC  National  Peer 
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Paragon  to  explain  drag  reduction  in  turbulent 
flow  by  means  of  "riblets,"  parallel  grooves 
on  the  surface  of  an  object  moving  through 
the  flow.  Such  grooves  are  used  on  aircraft, 
in  pipelines,  and  on  racing  cars  and  sleds,  to 
improve  performance.  The  group  also  mod- 
els flows  in  micro-electromechanical  systems 
used  in  surgery  and  other  complex  applica- 
tions, where  the  molecules  of  the  fluid  are  not 
much  smaller  than  the  channels  in  which  they 
flow.  Both  projects  are  resulting  in  new  ways 
to  optimize  the  performance  of  broad  classes 
of  machinery. 

Designing  Better  Aircraft 

Dino  Roman,  John  Vassberg,  and  Tom 
Gruschus  of  McDonnell  Douglas  are  using 
SDSC's  Cray  C-90  for  a  computational  fluid 
dynamics  simulation  of  an  aircraft  in  flight 
and  to  visualize  the  results  using  FAST 
(Flow  Analysis  Software  Toolkit)  on  a  Sili- 
con Graphics  IRIS  workstation.  Tracer  parti- 
cles are  released  into  the  flow  field  in  front  of 
the  aircraft  and  allowed  to  follow  the  stream- 
lines around  the  vehicle.  A  cutting  plane 
through  the  3D  volume  of  data  is  placed  to 
intersect  the  aircraft  fuselage  and  wings.  The 
aerospace  industry  relies  on  computational 
fluid  dynamics-the  simulation  of  air  or  fluid 
flow-to  design,  develop,  and  test  new  aero- 
nautical configurations.  This  process  enables 
companies  to  test  new  models  quickly  to  .se- 
lect candidates  for  wind-tunnel  testing.  Such 
methods  used  in  aircraft  design  and  manufac- 
ture can  give  American  companies  a  techno- 
logical edge  in  the  global  market. 

Crash  Testing  Street  Signs 

California  State  Department  of  Trans- 
portation (Caltrans)  engineer  Payam  Row- 
hani  and  SDSC  engineer  Chuck  Charman 
generated  a  computerized  model  of  a  crash- 
test  vehicle,  called  a  "bogie,"  on  the  SDSC 
Cray  Research  Y-MP  for  simulations  of  test 
crashes  with  sign  and  lighting  supports.  They 
fine-tuned  the  bogie  front-end  design  with  the 

Review  Board,  another  example  of  the  tight 
luiks  between  members  of  the  MetaCenter  and 
the  scientific  community. 


computer  model  to  minimize  the  number  of 
validation  tests  necessary  at  the  Federal  Out- 
door Impact  Laboratory,  operated  by  the 
Federal  Highway  Administration.  In  a  sec- 
ond application,  Charman  is  working  with 
Caltrans  engineers  William  Nokes  and  Dario 
Perdomo,  who  are  designing  pavement  using 
structural  modeling  techniques.  They  are  re- 
searching the  use  of  new  and  recycled  mate- 
rials and  these  materials'  response  to  differ- 
ent axle  and  tire  configurations.  They  are  us- 
ing the  supercomputer  and  the  visualization 
facilities  to  explore  new  truck  suspension 
systems,  and  new  tires  and  heavier  loadings, 
innovative  pavement  structures  with  recycled 
materials  and  rubber  and  polymer-modified 
binders.  The  results  of  this  research  are  ex- 
pected to  lead  to  significant  cost  savings  in 
the  design,  construction,  maintenance,  and 
rehabilitation  of  pavement  structures. 

Earth  Sciences  and  the  En- 
vironment 

From  understanding  the  motions  of  the 
Earth's  convective  mantle  to  daily  compu- 
tation of  air  pollution  levels  in  southern  Cali- 
fornia, the  resources  of  the  NSF  MetaCenter 
are  being  used  to  compute  and  visualize  the 
complexity  of  the  natural  world  around  us. 
The  US  Army  is  working  with  academics  to 
determine  how  they  can  practice  tank  maneu- 
vers without  endangering  the  breeding  habits 
of  the  sage  grou.se.  Pollution,  whether  under- 
ground or  in  the  air,  is  a  difficult  coupling  of 
chemical  reactions  and  flow  dynamics  which 
must  be  understood  in  detail  if  corrective 
measures  are  to  be  efficacious.  High  per- 
formance computers  also  act  as  time  ma- 
chines, allowing  for  faster-than-realtime 
computation  of  severe  storms.  Finally,  to 
improve  global  weather  or  climate  forecasts, 
supercomputers  allow  researchers  to  zero  in 
on  the  critical  coupling  physics  of  such  proc- 
esses as  mixing  at  the  air/ocean  interface. 

Detoxiflcation  of  Ground  Water 

Christine  Shoemaker  (Dept.  of  Civil  and  En- 
vironmental Engineering,  Cornell  University) 
has  been  a  pioneer  user  of  the  scalable  IBM 
SP  machines  at  the  CTC  for  the  development 
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of  numerically  efficient  supercomputer  algo- 
rithms for  optimal  control  of  dynamical  sys- 
tems and  the  application  of  these  techniques 
to  detoxification  of  contaminated  groundwa- 
ter. Her  efforts  are  leading  to  methods  of 
determining  the  most  cost-effective  way  to 
clean  up  the  groundwater  by  computing  time- 
varying  rates  of  pumping.  Shoemaker's 
group  has  also  developed  an  animation,  using 
the  visualization  resources  (both  hardware 
and  personnel)  of  the  CTC,  that  represents 
the  effects  of  different  policies  and  natural 
chemical  and  biological  processes  on 
groundwater  cleanup.  Such  animations  are 
crucial  for  conveying  the  results  of  basic  re- 
search to  the  mixed  audience  involved  in  set- 
ting environmental  policy. 

Sage  Grouse-Endangered  Species  and 
the  US  Army 

Working  with  the  U.  S.  Army,  Bruce  Han- 
non  (Dept.  of  Geography,  UIUC)  and  Jim 
Westervelt,  of  the  U.  S.  Army  Construction 
Engineering  Research  Laboratory,  have  de- 
veloped an  ecological  model  for  the  sage 
grouse,  an  endangered  species,  population  on 
an  army  training  base  in  Washington  State. 
Using  the  Macintosh  software  STELLA,  the 
CM-5  and  the  GRASS  geographical  infor- 
mation systems,  these  researchers  were  able 
to  optimize  the  scheduling  of  training  exer- 
cises to  maximize  grouse  reproduction  and 
longevity.  For  each  geographic  cell,  a  very 
large  STELLA  model  was  constructed,  rep- 
resenting the  grouse  at  various  life  stages, 
different  kinds  of  plants  and  predators,  soil 
type  and  moisture,  weather-all  the  many 
physical  variables-and  also  introduce  the  nec- 
essary human  activities  like  tank  and  troop 
maneuvers  on  the  army  range.  Each  cell,  of 
which  there  are  over  a  hundred  thousand  in 
the  GIS  covering  the  army  base,  could  con- 
tain 1 00  to  200  variables.  This  work  demon- 
strated the  efficacy  of  coupling  GIS  datasets 
to  ecological  models  and  running  them  in  a 
client-server  fashion  between  a  Macintosh 
and  the  CM-5. 


Storm  modehng/forecasting 

Robert  Wilhelmson  (Dept.  of  Atmospheric 
Sciences.  UIUC  and  NCSA)  and  his  col- 
leagues have  been  able  to  simulate  the  devel- 
opment of  tornadoes  embedded  within  larger 
storms  called  supercells  (producing  the  larg- 
est tornadoes)  and  along  low  level  con- 
vergence boundaries  (e.g.  along  a  thunder- 
storm cold  air  boundary)  using  both  tradi- 
tional vector  supercomputers  (NCSA  and 
PSC  Cray  Research  Y-MP  and  C90  and 
NCSA  TMC  CM-2  and  CM-5).  Study  of 
these  results  is  leading  to  a  better  under- 
standing of  when  tornadoes  will  develop  and 
to  more  accurate  tornado  warnings.  The  visu- 
alization of  the  intemal  dynamics  of  a  .severe 
thunderstorm,  created  by  the  NCSA  visuali- 
zation team  in  1989,  is  perhaps  the  most 
widely  viewed  visualization  of  a  supercom- 
puter simulation  ever  made.  It  had  a  major 
worldwide  impact  on  the  adoption  of  scien- 
tific visualization  as  a  working  tool  of  com- 
putational science. 

Kelvin  Droegemeier,  a  former  student  of 
Wilhelmson's,  and  his  colleagues  associated 
with  the  Center  for  Analysis  and  Prediction 
of  Storms  (CAPS),  an  NSF  S  &  T  Center, 
have  used  the  NCSA  and  PSC  Cray  Re- 
search supercomputers  to  develop  the  Ad- 
vanced Regional  Prediction  System,  a  com- 
putational model  for  forecasting  severe 
storms.  As  of  Spring  1994,  this  model  has 
been  used,  with  data  augmented  by  the  sin- 
gle-Doppler  radar  network  now  being  de- 
ployed by  NOAA,  in  daily  weather  reporting 
on  an  experimental  basis.  Because  of  their 
use  of  parallel  supercomputers  they  have 
shown  that  regional  storm  forecasts  based  on 
very  high  resolution  models  are  possible  with 
the  advent  of  teraflop  computing  capabilities 
in  the  next  few  years.  The  long-term  objec- 
tive is  to  improve  the  prediction  of  hazardous 
weather  on  scales  ranging  from  a  few  kilo- 
meters (an  individual  storm)  and  tens  of 
minutes  to  hundreds  of  kilometers  (a  squall 
line  or  other  mesoscale  system)  and  several 
hours. 
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Los  Angeles  Smog 

Gregory  J.  McRae  (Massachusetts  Institute 
of  Technology)  and  Armistead  Russell 
(Carnegie  Mellon  University)  have  developed 
the  most  comprehensive  model  of  smog 
formation  available.  Their  modeling  of  smog 
in  Los  Angeles  on  the  PSC  C90  showed  that, 
contrary  to  EPA  policy  at  the  time,  it  is  nec- 
essary to  control  nitrogen  oxide  emissions  as 
well  as  hydrocarbons  to  control  smog.  This 
work  formed  the  scientific  underpinning  for 
the  Air  Quality  Management  Plan  adopted  in 
1988-89  for  the  Los  Angeles  air  basin,  the 
most  stringent  such  plan  in  the  United  States. 
Their  modeling  also  showed  that  alternative 
vehicle  fuels,  methanol  in  particular,  repre- 
sent a  worthwhile  strategy  for  improving  ur- 
ban air  quality,  which  influenced  inclusion  of 
this  policy  in  the  1990  revisions  to  the  Fed- 
eral Clean  Air  Act.  This  work  is  being  ex- 
tended using  the  combination  of  the  PSC 
Cray  Research  C90  and  T3D  as  an  NSF 
Grand  Challenge.  The  first  Forefronts  of 
Large-Scale  Computation  award,  given  in 
1989,  recognized  McRae  for  this  work. 

These  pioneering  computations  are  leading  to 
practical  tools  for  states  to  predict  air  pollu- 
tion levels.  The  Modeling  and  Meteorology 
Branch  of  the  California  Air  Resources 
Board  joined  the  SDSC  Industrial  Partners 
program  in  1991.  They  are  running,  on  the 
SDSC  Cray  Research  C90,  the  Urban  Air- 
shed model  which  estimates  hourly  pollutant 
concentrations.  They  results  are  u.sed  to  es- 
timate maximum  pollutant  concentrations  or 
population  exposure  statistics  for  different 
emissions  controls. 

Upper  Ocean  Mixing 

Sidney  Leibovich  (Dept.  of  Mechanical  and 
Aerospace  Engineering,  Cornell  University) 
has  developed  a  mathematical  model  on  the 
CTC  IBM  ES/9000  and  SPl  that  uses  a  .se- 
ries of  equations  to  simulate  what  happens 
when  wind  blows,  waves  form,  and  Lang- 
muir  circulations  begin  to  mix  the  upper 
ocean.  This  mixing  has  a  marked  influence 
on  the  density  of  the  water;  the  ocean's  den- 
sity structure  alters  its  current  patterns,  and 


the  combined  phenomena  influence  the  ex- 
change of  heat  between  the  ocean  and  the  at- 
mosphere. All  of  this  has  environmental  and 
ecological  consequences,  and  finding  ways  to 
predict  this  chain  of  events  is  of  interest  to 
ecologists.  oil  companies,  meteorologists, 
climatologists,  undersea  communications 
experts,  and  government  policy  makers. 

Simulating  Climate  using  Distributed 
Supercomputers 

C.  Roberto  Mecho.so,  University  of  Cal- 
ifornia, Los  Angeles;  Larry  Bergman,  Jet 
Propulsion  Laboratory;  Carl  Scarbnick,  Gary 
Hanyzewski,  and  Bilal  Chinoy,  SDSC;  Paul 
Messina,  California  Institute  of  Technology; 
and  Robert  Malone  and  Rick  Smith,  Los 
Alamos  National  Laboratory  are  developing  a 
coupled  general  circulation  model  distributed 
over  a  heterogeneous  network.  The  atmos- 
pheric component,  which  runs  on  JPL's 
CRAY,  is  a  finite-difference  model  with 
state-of-the-art  parameterizations  of  convec- 
tion, planetary  boundary  layer  processes,  and 
radiation.  It  provides  wind  stress  and  heat 
flux  information  to  the  finite-difference  oce- 
anic component,  which  runs  on  Caltech's 
Delta  or  SDSC's  Paragon.  The  oceanic  com- 
ponent then  returns  the  sea  surface  tempera- 
ture to  the  atmospheric  component.  Results 
of  the  coupled  system  show  a  very  realistic 
seasonal  cycle.  This  is  a  significant  step  in  the 
development  of  a  coupled  system  distributed 
across  heterogeneous  computer  environ- 
ments, a  mam  goal  of  the  CASA  gigabit  test- 
bed  project. 

Planetary  Sciences,  astronomy, 
AND  Cosmology 

The  sciences  of  the  world  beyond  the  Earth 
have  always  been  an  interaction  of  ob- 
servation of  unexpected  events  and  theory 
creating  a  model  built  on  the  laws  of  physics 
and  chemistry  which  explain  the  observa- 
tions. Therefore,  it  is  proper  to  see  high  per- 
formance computing  and  communications 
making  major  impacts  on  both  observation 
and  theory.  Indeed,  as  we  saw  in  the  recent 
impact  of  Comet  Shoemaker-Levy  9  with 
Jupiter,  the  observatories  on   earth  and    in 
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space  have  become  intimately  linked  with 
each  other  and  with  the  theoretical  simula- 
tions. Supercomputers  are  becoming  inte- 
grated into  observational  facilities,  like  the 
Grand  Challenge  BIMA  millimeter  observa- 
tory, and  with  observational  programs  like 
the  ones  which  have  led  to  discovery  of  new 
millisecond  pulsars  or  the  first  extrasolar 
system  planets,  themselves  orbiting  a  pulsar. 

The  ability  of  numerical  methods  to  solve 
even  the  most  complex  of  fundamental 
physical  laws,  such  as  Einstein's  equations 
of  General  Relativity,  is  leading  to  a  very 
rapid  understanding  of  the  dynamics  of 
strong  field  events,  such  as  the  collision  of 
black  holes.  In  perhaps  the  grandest  .scale 
challenge  possible,  the  universe  itself  is  a 
subject  of  investigation  by  a  several  Grand 
Challenge  teams  using  resources  of  the 
MetaCenter  to  discover  how  the  large  scale 
structures  in  the  universe  evolved  from 
nearly  perfect  homogeneity  at  the  time  of  the 
formation  of  the  microwave  background. 

Comet  Collision  with  Jupiter 

Mordecai-Mark  Mac  Low  (Dept.  of  As- 
tronomy &  Astrophysics,  Univ.  of  Chicago 
and  Dept.  of  Astronomy,  UIUC)  and  Kevin 
Zahnle  (Space  Science  Division,  NASA 
.Ames  Research  Center)  simulated  the  impact 
of  Comet  Shoemaker-Levy  9  on  Jupiter  us- 
ing the  astrophysical  gas  dynamics  code 
ZEUS-3D,  developed  at  the  NCSA,  running 
on  the  PSC  Cray  Research  C90.  Their  simu- 
lations, made  months  before  the  actual  im- 
pact, were  widely  used  by  planetary  as- 
tronomers to  prepare  for  their  observing 
runs.  Their  results  showed  that  the  comet 
would  penetrate  only  shallowly  into  the 
planet's  atmosphere  before  exploding  and 
that  the  explosion  would  break  out  of  the  at- 
mosphere. Their  animations  were  available 
using  the  global  MosaicAVWW  infrastruc- 
ture and  received  worldwide  coverage  in  both 
the  popular  and  scientific  press. 


Discovery  of  First  Extrasolar  System 
Planet 

A  statistical  analysis  by  Alexander  Wol- 
szczan  (Dept.  of  Astronomy  and  Astro- 
physics, Penn  State  Univ.)  of  data  received 
from  the  radio  telescope  at  Arecibo  Ob- 
servatory in  Puerto  Rico  led  him  to  theorize 
the  existence  of  planets  orbiting  around  a  pul- 
sar. Searching  millions  of  bits  of  informa- 
tion using  the  supercomputer  facilities  at  the 
CTC,  he  detected  an  unusual  complexity  in 
the  pattern  of  the  pulses'  arrival  time.  The 
behavior  suggested  that  the  pulsar's  motion 
is  affected  by  the  presence  of  other  orbiting 
objects.  Applying  a  perturbation  model  pro- 
posed by  Frederic  Rasio  at  Princeton  Univer- 
sity, along  with  Philip  Nicholson,  Stuart 
Shapiro,  and  Saul  Teukolsky  at  Cornell  Univ. 
to  a  new  set  of  observations,  Wolszczan 
found  that  it  predicted  the  effect  that  two  or- 
biting planets  would  have  upon  each  other  as 
they  moved  through  space.  The  model  fits, 
and  Wolszczan  can  be  credited  with  identify- 
ing the  first  planetary  system  beyond  our 
own. 

The  April  22  1994  issue  of  Science  featured 
this  work,  with  front  page  coverage  of  news 
also  in  the  New  York  Times  that  week  as  well 
as  several  other  major  metropolitan  newspa- 
pers, television,  and  popular  magazines. 
Wolszczan  continues  to  explore  this  new 
world  using  parallel  capabilities  of  the  IBM 
SP  machines  at  the  CTC. 

Building  the  BIMA  Radio  Telescope 

This  HPCC  Grand  Challenge  project  at 
NCSA  is  implementing  a  prototype  of  the 
next  generation  of  astronomical  tele.scope 
systems  -  remotely  located  telescopes  con- 
nected by  high-speed  networks  to  very  high 
performance,  scalable  architecture  computers 
and  on-line  data  archives,  which  are  accessed 
by  astronomers  over  Gbit/sec  networks.  Data 
taken  by  the  BIMA  telescope  in  northern 
California  are  now  transferred  in  real  time  to 
NCSA  (using  a  combination  of  high-speed 
leased  line  and  NSFnet),  entered  into  a  data- 
base program,  and  archived  on  the  NCSA 
mass   storage  system.    Astronomers    locate 
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data  in  the  archive  and  load  data  onto  NCSA 
or  remote  computers  for  processing  using  the 
NCSA  Mosaic  software.  Also,  a  digital  li- 
brary of  fully  processed  radio  images,  which 
will  mclude  an  all-sky  radio  survey  created 
by  the  NRAO's  VLA  with  a  size  of  1 
terabyte,  is  being  developed.  One  of  the  ma- 
jor activities  that  BIMA  and  NCSA  staff  are 
involved  in  is  the  creation  of  a  new  standard 
software  package  AIPS-i-i-,  written  in  C++ 
for  extensibility  and  maintainability,  to  re- 
place the  older  standard  packages  AIPS, 
which  was  written  in  FORTRAN.  The 
AlPS-t-t-  software  is  being  developed  by  an 
international  collaboration  including  BIMA 
member  institutions,  the  National  Radio  As- 
tronomy Ob.servatory.  and  scientists  from 
Australia.  Canada.  England,  the  Netherlands 
and  India.  With  AIPS-)-+,  an  astronomer  can 
learn  the  system  once  and  then  process  im- 
ages on  nearly  any  computer  using  data  from 
almost  any  radio  telescope.  NCSA  will  im- 
plement computationally  intensive  routines 
on  the  SGI  Power  Challenge.  Finally,  discus- 
sions are  underway  to  have  the  BIMA  tele- 
scope array  be  part  of  the  development  of  a 
very  large  national  millimeter  array  radio 
telescope,  in  cooperation  with  the  Caltech  and 
the  National  Radio  Astronomy  Observatory. 

FuLsar  Searching  and  Discovery 

Joseph  Taylor  (Dept.  of  Astrophysical  Sci- 
ences, Princeton  Univ.  and  1993  Nobel  prize 
winner  in  Physics)  and  his  colleagues  used 
the  PSC  C90  to  aid  in  the  search  for  pulsars 
by  analyzing  radio-frequency  data  gathered  at 
the  Arecibo  Observatory.  Since  using  su- 
percomputing  to  sift  through  the  enormous 
amount  of  data  obtained,  this  approach  has 
resulted  in  the  discovery  of  20  previously 
unknown  pulsars,  including  five  millisecond 
pulsars. 

Accretion  Disks  Around  Black  Holes 

Most  galaxies  .seem  to  harbor  massive  black 
holes  (between  one  million  and  ten  billion 
solar  masses)  at  their  centers.  When  gas  ac- 
cretes onto  these  holes  enormous  energies 
can  be  released,  turning  the  most  luminous  of 
these  objects   into  quasars.      John   Hawley 


(Dept.  of  Astronomy.  Univ.  of  Virginia)  has 
been  using  supercomputers  for  over  a  decade 
to  investigate  the  accretion  disks  which  form 
around  black  holes.  Using  the  Cray  Research 
Y-MP  and  C90  at  NCSA  and  PSC.  Hawley 
developed  a  new  magnetohydrodynamic 
(MHD)  code  which  enabled  him  and  Steven 
Balbus  to  explore  the  nature  of  a  powerful 
MHD  instability  in  accretion  disks.  They 
discovered  that  this  almost  universal  instabil- 
ity is  the  underlying  physical  mechanism 
which  gives  rise  to  the  effective  turbulent  vis- 
cosity in  the  disk,  which  previously  theorists 
had  only  modeled  parametrically.  For  this 
discovery.  Hawley  received  the  1993  Helen 
B.  Warner  Prize  from  the  American  As- 
tronomical Society. 

Black  Hole  Collision  Dynamics 

The  two-body  problem  in  General  Relativity 
is  represented  by  the  motion  of  two  black 
holes.  In  contra.st  to  the  two-body  problem 
in  Newtonian  gravity  which  was  solved  in 
closed  form  over  300  years  ago.  the  orbiting 
motion  of  two  black  holes  is  an  outstanding 
research  frontier,  recently  being  funded  by 
the  NSF  as  a  Grand  Challenge.  Two  re- 
search teams  in  this  Grand  Challenge.  Ed 
Seidel,  NCSA  Director  Larry  Smarr.  and 
coworkers  (Dept.  of  Physics  and  NCSA, 
UIUC)  and  Stu  Shapiro  and  Saul  Teukolsky 
(Dept.  of  Astronomy  and  Physics,  Cornell 
University)  have  used  a  variety  of  supercom- 
puters at  NCSA.  PSC.  and  CTC  to  compute, 
for  the  first  time,  the  evolution  of  the  event 
horizon  representing  the  head-on  collision  of 
two  black  holes.  The  spacetime  diagram  of 
the  collision  shows  a  picture  like  the  pre- 
dicted "pair  of  pants",  analyzed  qualitatively 
by  Stephen  Hawking  and  others  in  the  early 
1970s,  but  the  supercomputer  computation 
gives  for  the  first  time  quantitative  details 
about  the  area  increase  and  the  caustics 
formed  as  the  null  geodesies,  which  generate 
the  horizon,  intersect.  Seidel  and  coworkers 
show  that  in  general  such  violent  deforma- 
tions of  black  holes  cause  the  horizon  to  os- 
cillate with  the  quasinormal  modes  previ- 
ously studied  only  in  linear  perturbatioiT  the- 
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ory.  The  Grand  Challenge  team  is  now  mov- 
ing on  to  the  full  spiraiing-in  situation,  which 
is  exjjected  to  be  one  of  the  first  events  ob- 
served by  the  LIGO  gravitational  wave  ob- 
servatory late  in  this  decade. 

Largest  cosmological  simulation 

This  simulation,  carried  out  by  Michael 
Norman  and  Greg  Bryan  (NCSA  and  Dept. 
of  Astronomy),  on  the  TMC  CM-5  is  the 
first  which  is  sufficiently  accurate  to  allow 
direct  comparison  with  x-ray  telescopic  ob- 
servations, thereby  allowing  a  critical  test  of 
this  popular  cosmological  model.  A  block  of 
the  universe  half  a  billion  light  years  on  a  side 
was  evolved  from  the  era  of  the  formation  of 
the  microwave  background  (with  tiny  inho- 
mogeneties  as  quantified  by  the  COBE  satel- 


lite) until  the  present.  A  single  evolution  re- 
quired 30  hours  on  the  512-node  CM-5  and 
over  14GB  of  memory  to  hold  the  model. 
The  results  were  announced  at  the  May  1994 
meeting  of  the  American  Astronomical  Soci- 
ety and  covered  in  the  popular  press  by  CBS- 
TV  and  the  Washington  Post.  The  computa- 
tion is  one  of  a  number  now  being  made  by 
members  of  the  Grand  Challenge  Cosmol- 
ogy Consortium  using  resources  of  the 
MetaCenter.  Jerry  ©.striker  (NAS),  Chair  of 
the  Department  of  Astrophysical  Sciences  at 
Princeton  University  is  The  largest  simula- 
tion of  the  formation  of  x-ray  clusters  in  a 
universe  dominated  by  a  mixture  of  cold  and 
hot  dark  matter  was  carried  out  using 
NCSA's  512  node  CM-5.  PI  of  the  GC-"*. 


Evolution  of  the  MetaCenter  Concept 


As  the  decade  of  the  1990s  opened,  it  became 
clear  that  we  were  witnessing  fundamental 
changes  in  the  way  researchers  use  high- 
performance  computing  and  communications 
systems,  driven  by  technological  advances  in 
computers  and  networking.  Anticipating 
these  changes,  the  four  NSF  supercomputing 
centers  formed  a  collaboration  based  on  the 
concept  of  a  national  MetaCenter  for  compu- 
tational science  and  engineering:  a  collective 
of  intellectual  and  physical  resources  unlim- 
ited by  geographical  or  technological  con- 
.straint. 

The  primary  objective  was  to  realize  ways  in 
which  researchers  could  be  best  served  and 
their  problems  directed  to  that  architecture  or 
combination  of  architectures  best  suited  to 
their  solution,  on  a  nationwide  basis.  Thus 
the  MetaCenter.  begun  in  1992,  was  con- 
ceived as  a  "garden  of  architectures"  that 
could  demonstrate  ways  of  providing  na- 
tionwide resources  to  support  scientific  and 
technological  advances  and  supply  a  frame- 
work within  which  the  NSF  centers  program 
could  move  into  the  next  era  of  high- 
performance  computing.  In  1994.  NCAR's 
Scientific  Computing  Division  joined  the 
MetaCenter.  In  addition,  the  NSF  established 


a  program  of  MetaCenter  Regional  Affiliates 
(MRA)  under  which  other  institutions  could 
pursue  projects  of  interest  in  collaboration 
with  MetaCenter  institutions;  MetaCenter 
sites  were  involved  with  several  MRA  pro- 
posals. 

MetaCenter  projects  involve  the  use  of  the 
resources  and  expertise  available  at  more  than 
one  MetaCenter  site.  Often  these  projects  are 
grand  challenges  or  national  challenges- 
many  have  been  highlighted  el.sewhere  in  this 
report. 

In  addition  to  direct  support  of  researchers, 
the  MetaCenter  sites  have  joined  to  work  on 
specific  projects  of  benefit  to  the  .scientific 
community.  Projects  begun  by  the  Meta- 
Center during  the  first  two  years  include; 

•  National  File  System;  Separate  cells 
have  been  established  at  each  site  based  on 
the  Andrew  File  System  (AFS);  a  national 
cell  that  will  serve  as  the  basis  for  a  pro- 
duction system  is  being  implemented 
With  such  a  system  in  place,  users  will  be 
able  to  move  transparently  among  the 
MetaCenter  computational  systems. 
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National  Archival  Storage  System; 
Work  has  begun  on  a  proposed  archival 
storage  system  based  on  manufacturer- 
supported  and  MetaCenter-enhanced  prod- 
ucts, including  versions  of  UniTree  and 
the  Data  Migration  Facility  from  Cray  Re- 
search. With  such  a  system  in  place,  files 
on  any  center  machine  can  be  migrated 
transparently  among  other  center  ma- 
chines, the  national  file  system,  and  the  na- 
tional archival  .storage  system. 

Video  Collaborator/:  Using  video 
teleconferencing  equipment,  the  re- 
searchers and  staff  of  MetaCenter  in- 
stitutions meet  regularly,  often  with  par- 
ticipation from  the  NSF.  The  equipment 
has  been  used  to  broadcast  joint  training 
workshops  like  the  MetaCenter  Computa- 
tional Science  Institutes  in  Parallel  Com- 
puting conducted  at  SDSC  and  NCSA. 
Plans" are  to  install  additional  hardware  to 
strengthen  this  system. 

Visualization  Collaboration:  Meta- 
Center and  NSF  Science  and  Technology 
Centers  researchers  participated  in  a 
DASC-funded  MetaCenter  Visualization 
Workshop  held  at  SDSC  in  1993,  and  ex- 
periments have  begun  in  passing  visual- 
izations between  the  centers  in  real  time  to 


determine  bottlenecks  and  potential  prob- 
lems. 

Online  Information:  An  online  in- 
formation system  using  the  gopher  proto- 
cols has  been  implemented;  it  is  now  be- 
ing converted  to  be  accessible  via  World- 
Wide  Web  browsers.  Plans  are  to  broaden 
the  information  available  and  incoiporate 
new  technologies.  A  joint  Science  High- 
lights Repository  will  soon  include 
WWW  links  to  MetaCenter-supported  re- 
search projects  worldwide. 

•  Networking:  Plans  were  developed  to 
upgrade  the  high-speed,  wide-area  net- 
work connecting  the  centers  into  a  very 
high  speed  backbone. 

New  projects  for  the  MetaCenter  include  ex- 
ploring virtual  reality  for  use  in  the  scientific 
discovery  process,  u.se  of  the  basis  developed 
in  the  Online  Information  project  to  develop  a 
national  training  program  to  promote  under- 
standing and  expertise  in  the  use  of  parallel 
computing,  and  a  joint  evaluation  of  the 
scalable  parallel  computers  acquired  by  the 
NSF  centers.  A  project  has  also  been  initiated 
to  establish  a  security  research  and  response 
team  to  monitor  and  coordinate  responses  to 
incidents  of  .security  violations. 


Recognition  Accorded  NSF  supercomputer  Users  and  Projects 

1994  [in  part  for  this  work  performed  at  NSF 
centers].  David's  PhD  advisor  and  the  third 
Feenberg  awardee  is  Mai  Kalos,  Director  of 
the  Cornell  Center. 


The  previous  sections  on  Accomplishments 
of  the  NSF  Supercomputer  Program  cite 
several  major  ways  in  which  the  research  of 
the  centers  has  been  recognized.  Specific  no- 
table citations  and  users  are  summarized 
here,  with  the  complete  description  available 
in  the  appropriate  sections.  In  addition,  scien- 
tific visualizations  produced  at  the  centers 
have  provided  popular  recognition  as  covers 
of  scientific  magazines  and  technical  pro- 
grams, as  evidenced  by  numerous  covers  of 
Science,  and  NSF's  own  publication  Mosaic. 

Pha.se  Transitions  of  Solid  Hydrogen  ... 
David  Ceperley  was  awarded  the  fifth  Eu- 
gene Feenberg   Memorial   Silver  Medal    in 


Protein  Kina.se  solution  ...  (Susan  Taylor  et 
al)  won  the  Forefronts  of  Large-Scale  Com- 
putation Award  presented  at  Supercomputing 
'93  for  this  work... 

Kinking  DNA  ...  John  Rosenberg  won  the 
1991  Forefronts  of  Large-Scale  Computation 
award  for  this  research,  and  his  work  was 
cited  in  the  1993  Computerworld  Smith- 
sonian award  for  science  which  was  given  to 
the  Pittsburgh  Supercomputer  Center  (PSC). 
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Heart  Modeling  ...  This  research  by  Charles 
Peskin  won  the  1994  Computerworld  Smith- 
sonian award  tor  Breakthrough  Computa- 
tional Science.  Peskin  was  previously 
awarded  a  Mac  Arthur  Prize  Fellowship  in 
1983,  for  earlier  studies  in  heart  modelmg. 

Ultra-high-strength  Steels  Gregory  B.  Olson 
and  Arthur  J.  Freeman  (Northwestern  Uni- 
versity) use  computer  modeling  to  design 
ultra-high  strength  steel  for  weight-critical 
applications  ...  This  work  was  reported  in  the 
July  15.  1994  Science.  Freeman  and  his 
group  have  been  users  of  NSF  su- 
percomputers since  the  foundmg  of  the  pro- 
gram on  a  wide  range  of  problems  in  materi- 
als sciences.  Arthur  Freeman  received  the 
first  Materials  Research  Society  Medal  and 
the  first  Award  in  Magnetism  from  the  Inter- 
national Union  of  Pure  and  Applied  Physics. 

Los  Angeles  Smog  ...  The  first  Forefronts  of 
Large-Scale  Computation  award,  given  in 
1989,  recognized  Gregory  McRae  for  this 
work. 

Theory  of  High  Temperature  Super- 
conductors ...The  Nobel  Prize  in  Physics  in 
1 987  was  for  the  discovery  of  a  new  class  of 
high  temperature  superconductors.  Thou- 
sands of  research  papers  have  been  written 
about  the.se  unique  materials...  David  Pines 
(NAS  and  first  Feenberg  'Medalist)  and 
Philippe  Monthoux  (Department  of  Physics, 
UIUC)  used  the  NCSA  Cray  Research  Y- 
MP  to  carry  out  a  strong  coupling 
(Eliashberg)  calculation  of  the  normal  state 
properties  and  Tc  for  the  model  experiment- 
based  magnetic  interaction  between  quasipar- 
ticles... 

Crystallography...  Herbert  Hauptman 
(Medical  Foundation  of  Buffalo.  Inc.)  won  a 
Nobel  Prize  in  1985  for  development  of  the 
"direct  method"  of  protein  structure  de- 
termination from  X-ray  crystallographic  data. 
In  a  collaboration  with  Russ  Miller  (State 
University  of  New  York  at  Buffalo).  Haupt- 
man and  Miller  have  developed  a  numerical 
approach  that  extends  the  "direct  method"  of 
determining  molecular  structure  from  X-ray 


crystallographic  data  to  larger  molecules,  be- 
yond its  present  limit  of  about  100  atoms... 

Pulsar  Searching  and  Discovery  ...  Joseph 
Taylor  (Dept.  of  Astrophysical  Sciences. 
Princeton  and  a  1993  Nobel  prize  winner  in 
Physics)  and  his  colleagues  used  the  PSC 
C90  to  aid  in  the  search  for  pulsars  by  ana- 
lyzing radio-frequency  data  gathered  at  the 
Arecibo  Observatory.  Using  supercomputing 
to  sift  through  the  enormous  amount  of  data 
obtained,  this  approach  has  resulted  in  the 
discovery  of  20  previously  unknown  pulsars, 
including  five  millisecond  pulsars. 

Accretion  Disks  Around  Black  Holes  ...  John 
Hawley  (Dept.  of  Astronomy.  Univ.  of  Vir- 
ginia) has  been  using  supercomputers  for 
over  a  decade  to  investigate  the  accretion 
disks  which  form  around  black  holes.  Using 
the  Cray  Research  Y-MP  and  C90  at  NCSA 
and  PSC,  Hawley  developed  a  new  magne- 
tohydrodynamic  (MHD)code  which  enabled 
him  and  Steven  Balbus  to  explore  the  nature 
of  a  powerful  MHD  instability  in  accretion 
disks.  ...  For  this  discovery,  Hawley  re- 
ceived the  1993  Helen  B.  Warner  Prize  from 
the  American  Astronomical  Society. 

Alpha  Shapes,  Biomolecules,  and  Cosmol- 
ogy Alpha  shapes,  a  form  of  geometric  mod- 
eling developed  by  the  1993  Waterman 
Award  winner  Herbert  Edelsbruner  (Dept.  of 
Computer  Science,  UIUC)  and  NCSA  staff 
member  Ping  Fu.  focuses  on  the  formal  def- 
inition, construction,  and  measurement  of 
shapes  for  any  given  point  set  in  space,  [They 
are...]  useful  in  studying  water  molecules  re- 
siding inside  a  protein.  NCSA  u.sers  have 
discovered  other  related  applications  of  alpha 
shapes  by  applying  them  to  such  diverse 
fields  as  adaptive  grid  generation,  medical 
image  analysis,  visualizing  the  structure  of 
earthquake  data,  and  the  large-scale  structure 
of  the  universe. 

The  Rise  of  the  MosaicAVWW  Information 
Infrastruture  NCSA  developed  the  Mosaic 
user  interface  software  which  provides  point- 
and-click  access  to  the  diverse  information 
storage  protocols  of  the  Internet,  such  as 
World  Wide  Web  (WWW),  Gopher,  FTP. 
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and  WAIS....  In  1994.  NCSA  was  given  In- 
joworld's  Publisher's  Industry  Achievement 


Award. 


Acronyms 


AFS 

Andrew  File  System 

(developed  at  Carnegie-Melon 

CD 

University  and  named  for  its 

CFS 

benefactors,  Andrew  Carnegie 

CMRR 

and  Andrew  Melon) 

ARPA 

Advanced  Research  Project 
Agency  of  the  Department  of 

CNRI 

Defense  (previously  called 

CPU 

DARPA) 

CTC 

ASC 

Division  of  Advanced  Scien- 

CTSS 

tific  Computing,  Computer 

DASC 

and  Information  Sciences  and 

DEC 

Engineering  Directorate 

(CISE).  NSF 

DICE 

ATM 

Asynchronous  Transmission 

Mode  -  high  bandwidth 

DJMS 

transmission  technology  for 

DMF 

WAN  use 

DX 

BIMA 

Berkeley,  Illinois,  MAryland 
telescope 

EOSDIS 

BLANCA 

NSF  Gigabit  testbed  -  see 

CASA 

EPA 

C 

A  programming  language  de- 

veloped at  Bell  Labs 

FAST 

(probably  the  3rd  in  a  series) 

and  used  as  an  easily  portable 

FDDI 

systems  development  lan- 

guage. 

C++ 

An  Object  Oriented  extension 
ofC 

FORTRAN 

CAD 

Computer  Aided  Design 

CAPS 

Center  for  Analysis  and  Pre- 
diction of  Storms 

CASA 

NSF  Gigabit  testbed  -  see 

FTP 

BLANCA 

GC 

CAT 

Cornell  University's  Biotech- 

Giga 
GIS 

nology  Center  for  Advanced 

Technology 

CAVE 

Cave  Automated  Virtual  En- 
vironment 

Gopher 

CCMS 

Computational  Center  for 

Macromolecular  Structure  — 

HDF 

an  SDSC,  UCSD,  and 

Scripps  Research  Institute 
collaboration 
Compact  Disk 
Common  File  System 
Center  for  Magnetic  Record- 
ing Research 

Corporation  for  National  Re- 
search Interests 
Central  Processor  Unit 
Cornell  Theory  Center 
Cray  Time  Sharing  System 
see  ASC 

Digital  Equipment  Corpo- 
ration 

Distributed  Collaboration  En- 
vironment) 

Dynamic  Job  Mix  Scheduler 
Data  Migration  Facility  (Cray) 
Data  eXplorer 

NASA  Earth  Orbitting  Satel- 
lite Distributed  Information 
System 

Environmental  Protection 
Agency 

Flow  Analysis  Software 
Toolkit 

Fiber  Data  Distribution  Inter- 
face (a  100  Mps  network  me- 
dium) 

FORmula  TRANslator  -  Ear- 
liest high  level  programming 
language  —  still  in  common 
use  on  high  performance 
computers 

File  Transfer  Protocol 
Grand  Challenge 
One  Billion-  10^ 
Geographic  Information  Sys- 
tem 

Internet  Browser  developed  at 
the  University  of  Minnesota 
(Golden  Gophers) 
NCSA  Hierarchical  Data 
Format 
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HiPPI 

High  Speed  Peripheral  Paral- 
lel Interface 

NCAR 

HP 

Hewlett  Packard 

NCSA 

HPC++ 

An  object  oriented  varient  of 

the  C  language  for  High  Per- 

NIH 

formance  Computers 

Nil 

HPCC 

High  Performance  Com- 

puting and  Communications 

NOAA 

HPF 

High  Performance  FOR- 

TRAN 

NRAO 

HPSS 

High  Performance  Storage 

System 

NREN 

HTTP 

Hyper-Text  Transfer  Protocol 

for  the  Internet 

NSF 

IBM 

International  Business  Ma- 

NSL 

chines 

PICT 

IFS 

AFS  port  by  Univ.  of  Michi- 

gan 

PSC 

InterNIC 

Internet  Network  Information 

Center 

PVM 

lUPAP 

International  Union  of  Pure 

QCD 

and  Applied  Physics 

RAID 

IVEM 

Intermediate  Voltage  Electron 

Microscope 

REL 

LANL 

Los  Alamos  National  Labo- 

latory 

REU 

LAN 

Local  Area  Network 

LIGO 

Laser  Interferometry  Gravita- 
tional Observatory 

RISC 

LLNL 

Lawrence  Livermore  National 

ROM 

Laboratory 

SDF 

MacCHESS 

Cornell  High  Energy  Syn- 
chrotron Source  for  Macro- 

SDMIR 

molecular  Modeling 

SDSC 

MHD 

Magneto  HydroDynamics 

MILC 

MIMD  Lattice  Calculations 

SGI 

Collaboration 

SIGGRAPH 

MIMD 

Multiple  Instruction,  Multiple 
Data 

Mosaic 

Internet  Hypertext  Browser 

—  WWW  client  developed  at 

SIMD 

NCSA 

MPP 

Massively  Parallel  Processors 

STM 

MRA 

MetaCenter  Regional  Affiliate 

MRI 

Magnetic  Resonance  In- 
terferometry 

TCP/IP 

NAS 

National  Academy  of  Sci- 
ences 

Telnet 
Tera 

National  Center  for  At- 
mospheric Research 
National  Center  for  Su- 
percomputer Applications 
National  Institutes  of  Health 
National  Information  In- 
frastructure 

National  Oceanic  and  Atmos- 
pheric Administration 
National  Radio  Astronomical 
Observatory 

National  Research  and  Edu- 
cation Network 
National  Science  Foundation 
National  Storage  Lab 
A  popular  image  format  for 
computers 

Pittsburgh  Supercomputer 
Center 

Parallel  Virtual  Machine 
Quantum  ChromoDynamics 
Redundant  Array  of  In- 
expensive Disks 
Renaissance  Experimental 
Laboratory 

Research  Experience  for  Un- 
dergraduates (NSF  program) 
Reduced  Instaiction  Set 
Computer 
Read  Only  Memory 
Standard  Data  Format 
San  Diego  Microscopy  and 
Imaging  Resource 
San  Diego  Supercomputer 
Center 

Silicon  Graphics,  Inc. 
Special  Interest  Group  - 
Graphics  —  an  influential  and 
popular  visualization  organi- 
zation and  annual  trade  show 
Single  Instruction,  Multiple 
Data 

Scanning  Tunneling  Mi- 
croscope 

Tranmission  Control  Pro- 
tocolAnternet  Protocol 
Internet  Computer  logon 
Services 
One  trillion-  10'2 
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TMC 

UNICOS 

UNIX 


vBNS 


VPLA 


Thinking  Machines.  Corp. 
Cray'.s  Version  of  Unix 
Popular  portable  computer 
operation  system  developed  at 
Bell  Labs  -  reportedly  does 
not  stand  for  anything  -  al- 
though some  believe  that  it 
was  a  pun  on  Multics,  a  time 
sharing  systems  then  in  use  at 
MIT 

very  high  speed  Backbone 
Network  Service 
Visual  Programming  Lan- 
guage for  Animation 


VR 

Virtual  Reality 

WAIS 

Wide  Area  Information  Sys 

tem 

WAN 

Wide  Area  Networks 

WWW 

World  Wide  Web  -  a  set  of 

commications  protocols  for 

Hypertext  established  at 

CERN. 

XMP  &  YMP  -  versions  of  Cray  computer 
systems  —  vector  multi- 
processors— used  as  refer- 
ences for  computer  perform- 
ance 
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Mr.  SCHIFF.  Thank  you,  Dr.  Hayes. 

I  would  like  to  recognize  Members  for  questioning.  I  would  ap- 
preciate it  if  Members  would  try  to  stay  at  least  loosely  to  five  min- 
utes just  because  we  have  two  more  panels. 

I  am  going  to  recognize  myself  first.  I  have  one  question. 

Dr.  Hayes,  I  would  like  to  be  clear,  myself,  and  be  clear  for  the 
record.  As  you  well  know  of  course  there  are  right  now  four 
supercomputing  centers  funded  in  the  NSF  program. 

What  I  would  like  to  know  is.  Does  your  Task  Force's  report  spe- 
cifically recommend  more  such  centers,  less  such  centers?  Numeri- 
cally speaking.  I  know  you  made  several  recommendations  on  how 
to  network  and  so  forth,  but  in  terms  of  the  number  of  centers,  is 
there  anything  in  the  report  that  addresses  that? 

Dr.  Hayes.  As  the  Task  Force  looked  at  the  overall  program,  the 
advice  that  we  had  had  fi-om  the  Foundation  and  our  own  sense  of 
what  the  budget  environment  was  going  to  be  for  the  next  five  to 
ten  years,  suggested  that  in  order  to  have  a  balanced  program,  that 
it  may  be  necessary  to  downsize  the  total  number  of  partnership 
sites. 

I  would  also  mention  that  the  Task  Force  was  also  aware  that 
an  important  part  of  a  growing  component  of  the  program  is  the 
Metacenter  Regional  Affiliates. 

In  our  recommendations  about  the  partnership  centers,  we  recog- 
nized that  these  Metacenter  Regional  Affiliates  could  and  should  be 
brought  into  the  program  as  a  natural  evolution  of  the  centers  pro- 
gram. 

So  within  the  Task  Force  there  was  an  expectation  that  in  order 
to  fund  the  balanced  program  that  was  envisioned,  that  it  may  re- 
grettably be  necessary  to  reduce  the  number  of  centers. 

Mr.  SCHIFF.  But  at  the  same  time,  you  are  proposing  the  origina- 
tion or  inclusion  of  other  centers — I'm  sorry?  use  the  term? — 

Dr.  Hayes.  Metacenter  Regional  Affiliates. 

Mr.  SCHIFF.  Right. 

Dr.  Hayes.  Yes,  that  presently  are  a  part  of  the  program,  and 
they  are  loosely  coupled  to  the  existing  four  centers. 

Mr.  SCHIFF.  Right. 

Dr.  Hayes.  I  think  one  of  the  important  things  to  appreciate  is 
that  between  the  time  of  the  Branscomb  Committee  Report  and 
when  the  Task  Force  was  called  together,  there  were  three  impor- 
tant changes. 

One  had  to  do  with  the  expectations  with  respect  to  the  budget. 
The  Branscomb  Committee,  which  I  was  a  member  of,  was  working 
in  an  environment  in  which  there  was  some  hope  and  expectation 
there  would  be  significant  growth  in  the  budget  for  high-perform- 
ance computing. 

So  the  possibility  that  we  would  have  to  downsize  the  number  of 
centers  was  really  not  addressed  in  a  formal  way  as  being  driven 
by  constraints  on  the  overall  budget. 

The  other  is  a  full  appreciation  of  the  importance  of  having  a  bal- 
ance within  the  p3rramid;  that  the  mid-range  systems,  because  of 
important  developments  in  the  technology,  were  becoming  increas- 
ingly important. 

So  the  possibility  of  buying  from  major  commercial  vendors  mid- 
range  systems  that  were  upwardly  compatible  by  increasing  the 
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total  number  of  nodes  on  the  systems  really  became  a  reality  in 
that  very  short  period  of  time.  And  the  importance  of  building  on 
that  development  in  the  technology  was  very  much  before  us  as  the 
Task  Force  was  formed. 

Our  sense  is  that  it  was  important  to  recognize  that  there  were 
a  number  of  these  mid-range  centers  that  could  be  coupled  more 
effectively  into  the  program,  and  there  will  be  efficiency  and  cre- 
ativity that  would  come  from  that  formal  coupling  rather  than 
loose  coupling  as  exists  today. 

Mr.  SCHIFF.  Thank  you,  Dr.  Hayes. 

Congressman  Cramer? 

Mr.  Cramer.  Thank  you.  I  apologize  for  having  to  be  out  of  the 
room  for  a  few  minutes. 

Welcome,  both  of  you,  and  thank  you  for  your  testimony. 

Dr.  Hayes,  are  there  any  significant  aspects  of  the  recommenda- 
tions of  your  committee  that  are  not  being  followed  by  NSF  in  re- 
structuring the  supercomputer  centers  program? 

Dr.  Hayes.  I  am  not  aware  of  any  major  elements  that  are  not 
reflected  in  the  program  announcement. 

There  are  other  issues  with  respect  to  the  high-performance  com- 
puting that  we  expect  that  the  Foundation  will  follow  through  on, 
such  as  taking  the  leadership  with  other  agencies  in  high-perform- 
ance computing  and  continuing  to  look  at  options  to  fine-tune  the 
review  process  for  allocation  at  the  centers,  but  those  are  not  mani- 
fest specifically  in  the  program  announcement. 

Mr.  Cramer.  Dr.  Young,  the  solicitation  for  the  new  centers  pro- 
gram requires  submission  of  a  preproposal 

Dr.  Young.  Yes. 

Mr.  Cramer.  So  you  are  going  to  have  a  preproposal,  if  I  can  say 
it  right,  and  a  final  proposal  as  well.  Will  there  be  overlap  between 
reviewers  of  those  two  sets  of  proposals? 

Dr.  Young.  I  would  certainly  expect  overlap.  They  may  not  be 
exactly  the  same  group,  but  I  would  expect  very  significant  overlap, 
yes. 

Mr.  Cramer.  So  you  would  expect  that  that  would  be  monitored 
so  that  we  don't  have  problems  as  a  consequence  of  trying  to  form 
what  you  do  around  one,  and  then  there  not  being  continuity  be- 
tween the  final  proposal  and  the  preproposal  reviewers? 

Dr.  Young.  The  intent  of  the  preproposal  stage  is  to  allow 
preproposers  to  come  in  and  get  advice  about  the  quality  of  their 
proposals;  to  get  input;  and  to  not  use  the  preproposal  rankings  as 
a  determinant  factor  in  what  the  final  will  be,  but  rather  as  a 
mechanism  for  giving  advice  to  people  coming  in. 

Mr.  Cramer.  And  I  want  to  say,  I  like  that  idea.  That  is  what 
I  had  hoped  I  would  hear,  because  I  think  that  is  a  good  way  to 
explore  it  through  that  process  that  work  with  and  try  to  cooperate 
and  encourage  that. 

Thank  you,  Mr.  Chairman.  That's  all  I  have. 

Mr.  SCHIFF.  Mr.  Ehlers,  you  are  recognized  for  five  minutes. 

Mr.  Ehlers.  Thank  you,  Mr.  Chairman. 

My  question  is  somewhat  tangential  and  I  hope  you  do  not  think 
this  reflects  on  your  report,  but  I  think  your  testimony  and  the  re- 
port speak  for  themselves. 
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We  have  a  particular  problem  in  the  Federal  Government  in 
terms  of  the  purchase  and  procurement  of  computers.  The  procure- 
ment process  itself  is  extremely  complex  because  of  the  paranoia  of 
the  public  over  the  years  that  someone  might  be  given  favored  con- 
tracts, or  some  such,  and  it  has  reached  the  point  where,  when  the 
Federal  Government  wishes  to  purchase  a  new  computer  using  the 
process  that  exists  there  is  an  absolute  guarantee  that  the  com- 
puter will  be  obsolete  by  the  date  it  is  delivered. 

I  have  two  questions  based  on  that. 

Number  one.  Is  that  a  problem  in  any  of  the  centers?  Are  you 
in  any  way  bound  by  any  of  the  procurement  requirements,  either 
directly  or  indirectly,  that  might  be  imposed  through  NSF? 

And  secondly.  Do  you  have  any  advice  to  give  us,  particularly 
since  I  am  looking  at  ways  of  restructuring  the  federal  procurement 
policies  vis-a-vis  computers,  do  you  have  any  advice  on  how  I  could 
proceed  or  how  we  could  proceed  in  order  to  eliminate  the  problem 
we  have  now? 

I  would  welcome  comments  from  either  one  of  you. 

Dr.  Young.  Well  I  will  take  that  first,  and  I  may  turn  to  Bob 
Borchers  if  he  has  backup  comments  he  wants  to  maike. 

First  of  all,  we  do  cooperative  agreements  with  vendors.  We  are 
on  the  leading  edge,  so  we  are  not  bound  by  the  federal  procure- 
ment process  in  acquiring  the  leading  edge  computers  here. 

What  we  typically  do  is  partner  with  vendors  to  bring  in  very 
early  versions  of  machines  that  are  often  very  interested  in  coop- 
eratively working  with  us  on  software  development  for  scientific 
applications  because  that  is  not  the  primary  drive  that  is  driving 
them  for  their  mass  market. 

So  we  have  a  mutual  interest  in  doing  software  development  and 
early  development  on  machines  that  enables  us  to  get  very  com- 
petitive rates  in  what  we  do  in  leasing. 

So  the  flat  answer  to  your  question  is:  we  simply  are  not  bound 
by  the  procurement  process,  and  it  works  very  well  to  partner  with 
industry  to  bring  these  machines  in  for  early  testing  and  develop- 
ment. 

With  regard  to  the  second  question.  No,  I  do  not  have  advice  on 
what  to  do  about  the  procurement  problems.  I  recognize  them. 

Mr.  Ehlers.  Thank  you. 

Dr.  Hayes? 

Dr.  Hayes.  Yes.  I  would  echo  the  comment  about  the  efficiency 
of  the  system.  I  think  that  many  center  directors  that  are  in  the 
Federal  Government  just  marvel  at  what  the  center  directors — at 
what  our  NSF  Supercomputing  Centers  have  been  able  to  do  in 
terms  of  getting  leading  edge  equipment  on  site  and  doing  it  in  a 
very  short  time  frame. 

The  key  point  that  Paul  mentioned  that  I  think  it  is  important 
to  recognize  is  the  extent  of  partnerships  between  the  centers  and 
the  vendors.  I  think  that  has  been  a  key  element  in  the  success  of 
the  program  right  from  the  beginning. 

I  think  one  of  the  new  elements  in  all  of  this  that  is  important 
to  keep  in  mind  is  that  the  recapitalization  cycle  for  the  new  hard- 
ware is  getting  shorter.  One  of  the  factors  that  went  into  our  con- 
sideration as  to  how  much  budget  would  be  required  to  support  a 
center  was  part  of  the  experience  from  the  history  of  the  program 
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that  if  you  run  with  long  recapitahzation  cycles  on  leading  edge 
technology,  that  it  leads  to  problems  within  the  program. 

So  that  the  center  directors  themselves  may  have  some  addi- 
tional insights  on  this  particular  question,  but  I  think  as  long  as 
the  resources  are  there  in  a  timely  way  and  importantly  they  do 
not  make  these  very  long-term  commitments  that  prevent  them 
from  recapitalization  on  an  appropriate  time  scale,  which  possibly 
in  this  environment  is  three  or  four  years,  why  you  can  see  the 
kind  of  turnover  in  a  timely  way  that  is  very  important  to  the  suc- 
cess of  this  program. 

Mr.  Ehlers.  And  do  you  have  any  advice  for  us? 

Dr.  Hayes.  No,  I  would  not  hazard  advice. 

[Laughter.] 

Mr.  Ehlers.  We  may  need  a  Hayes'  test  for  a  federal  computer 
system. 

Dr.  Young,  did  you  have  anjrthing  to  add? 

Dr.  Young.  No. 

Mr.  Ehlers.  Thank  you,  very  much. 

Mr.  SCHIFF.  Mr.  Doyle? 

Mr.  Doyle.  Thank  you,  Mr.  Chairman. 

Let  me  see  if  I  have  got  this  right.  Now  the  principal  role  of  the 
new  centers  program  is  to  provide  access  to  high-end  computing  in- 
frastructure for  the  academic,  science  and  engineering  community? 

Is  that  correct? 

Dr.  Hayes.  That  is  correct. 

Mr.  Doyle.  And,  Mr.  Hayes,  your  report  indicated  that  the 
supercomputers  at  the  existing  center  are  oversubscribed  by  a  fac- 
tor of  at  least  two,  and  the  demand  is  growing. 

Most  of  us  here  understand  that  little  or  no  funds  are  available 
for  the  restructured  program  which  must  fund  many  regional  cen- 
ters and  research  consortia  in  addition  to  each  leading  edge 
supercomputer  site,  however  many  there  are,  one  or  two;  and,  fi- 
nally, ARPA  is  not  likely  to  provide  support  to  the  new  centers  for 
hardware  upgrades  as  has  been  the  case  for  the  current  centers. 

Therefore,  since  the  new  program  will  be  increased  significantly 
in  scope  without  additional  funding,  why  do  you  believe  the  new 
program  will  adequately  support  high-end  computer  users? 

Dr.  Young.  I  don't  believe  it  will  adequately  support  high-end 
computer  users.  If  you  look  across  the  Foundation  as  a  whole,  per- 
haps one  in  four  of  the  proposals  for  research  that  come  into  the 
Foundation  that  are  good  proposals  are  in  fact  funded.  So  these  are 
tight  times  and  Chairman  Schiff  reminded  us  of  that  at  the  begin- 
ning. We  are  making  tough  choices. 

Dr.  Hayes.  I  would  say  that  part  of  the  perspective  that  the  Task 
Force  brought  to  the  table  was  the  realization  that  there  is  a  very 
large  amount  of  important  computational  research  that  can  be  done 
on  mid-range  systems  and  work  stations. 

The  view  of  the  Centers  Program  is  not  to  provide  support  for 
the  vast  majority  of  computational  studies.  The  NRC  report  on 
the — the  Brook-Sutherland  Report  talked  about  the  Centers  Pro- 
gram and  the  highest  end  computing  in  the  context  of  a  time  ma- 
chine; that  the  work  that  is  being  done  on  these  leading  edge  sys- 
tems that  are  very  expensive,  that  kind  of  work  is  basically  trail 
blazing  work  developing  algorithms  and  methodology,  testing  new 
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physics  and  chemistry  and  mathematics,  that  will  be  used  widely 
on  mid-range  machines  in  the  next  five  to  ten  years. 

Why?  Because  the  technology  capability,  the  underlying  tech- 
nology capability  is  doubling  every  18  to  24  months.  So  today's 
work  stations  basically  are  equivalent  to  the  supercomputers  that 
were  used  in  the  early  stage  of  the  program. 

So  from  a  national  perspective,  I  think  the  most  important  aspect 
of  this  program  is  to  provide  a  cadre  of  scientists  in  many  fields 
with  access  to  the  highest  end.  Not  that  it  is  going  to  be  the  total 
effort  in  the  Nation,  facilitating  the  total  effort,  in  high-perform- 
ance computing,  but  it  is  going  to  be  providing  the  cadre  of  the  best 
and  brightest  students  that  are  going  to  be  doing  trailblazing  re- 
search, but  there  is  still  going  to  be  the  vast  majority  of  the  work 
that  is  going  to  be  done  on  mid-range  machines  and  on  work  sta- 
tions. 

That  was  one  of  the  important  points  that  was  made  in  the 
Branscomb  Committee  Report,  and  I  think  that  is  being  borne  out 
by  what  is  happening  in  many  fields  today. 

Mr.  Doyle.  I  have  one  more  question. 

I  am  just  curious,  and  a  lot  of  us  are  just  trying  to  understand, 
you  know,  what  the  driving  forces  are.  We  hear  all  the  reports  on 
the  four  centers.  They  have  all  been  glowing  reports,  and  a  lot  of 
us  wonder  why  can  we  not  just  make  modifications  at  the  four  ex- 
isting centers  driving  this  movement  to  make  these  significant 
changes? 

Would  it  be  fair  to  say  that,  as  you  look  at  the  budget — you 
know,  we  all  understand  the  budget  climate  that  we  are  in  right 
now.  I  guess  what  I  am  trjdng  to  understand  is.  Is  it  dollars  that 
are  driving  these  decisions  to  look  at  maybe  going  from  these  four 
centers  to  maybe  one,  maybe  two? 

Why  could  we  not  do  this  by  modifying  what  goes  on  at  the  four 
existing  centers  and  spare  all  the  expense  of  a  recompete? 

What  is  driving  this? 

Dr.  Young.  I  will  take  that,  or  attempt  to  take  it. 

It  is  certainly  not  being  just  driven  by  dollars,  though  that  cer- 
tainly drives  part  of  it.  If  you  have  a  constant  budget  and  you  allow 
for  inflation,  it  gets  hard  to  do  as  much  as  you  have  done  in  the 
past  on  a  fixed  budget. 

But  as  Ed  pointed  out,  the  technology  is  also  driving  the  struc- 
ture of  the  new  program.  There  are  two  parts  to  that  technology. 

One  is  the  increasing  viability  of  mid-level  parallel  machines 
which  scale  up  to  the  high  end,  as  he  mentioned.  That  enables  new 
regional  and  state  centers,  for  example,  or  groups  at  universities  to 
partner  with  centers  in  a  sjmergistic  fashion  to  make  use  of  the 
technology  in  a  more  effective  fashion. 

We  think  that  this  new  program  will  use  that  technology  and  a 
corresponding  increase  in  high-band  width  connections  which  will 
enable  these  centers  to  partner  together  in  a  more  effective  way  to 
restructure  the  program  so  that,  although  we  will  have  fewer  high- 
end  leading  centers,  we  will  have  the  totality  of  more  full  partners 
in  the  program  and  we  think  we  will  get  greater  efficiency  with 
that. 

Mr.  Doyle.  Dr.  Hayes? 


92 

Dr.  Hayes.  I  think  the  concept  that  we  are  recommending — 
namely,  the  partnership  centers — is  right,  irrespective  of  whether 
the  number  is  two,  three,  four,  or  five.  So  let  me  be  clear  about 
that. 

Just  two  amplifying  insights  here: 

One  is  the  realization  that  the  very  best  people  in  computer 
science  and  in  computational  mathematics  and  in  computational 
engineering  generally  are  not  all  located  at  one  place.  So  that  the 
partnership  centers,  as  we  have  seen  with  the  Metacenter  Regional 
Affiliates,  play  a  very  important  role  in  the  training,  and  develop- 
ment, and  the  inclusion  of  a  much  broader  cross-section  of  talented 
people  within  the  concept  of  the  program. 

So  the  model  facilitated  by  the  technology  we  believe  to  be  the 
correct  model;  and  that  the  effort  to  coordinate  the  development 
and  the  acquisitions  and  the  allocation  of  times  is  well  worth  the 
effort  from  a  national  perspective. 

The  other  point  that  is  of  course  present  here  is  that  the  tech- 
nology is  changing  quite  rapidly.  And  that  if  you  do  not  keep  these 
leading  edge  sites  one  to  two  orders  of  magnitude  ahead  of  what 
you  can  do  with  a  work  station,  the  attractiveness  and  the  con- 
tributions that  will  be  forthcoming  from  these  centers  will  be  sig- 
nificantly reduced. 

So  you  have  these  two  things  that  are  tugging  at  each  other  and 
are  important  changes  in  the  computing  environment  from  what  it 
was  at  the  time  of  the  initiation  of  the  Centers  Program. 

Mr.  Doyle.  Thank  you,  Mr.  Chairman. 

Mr.  Ehlers  [presiding] .  Thank  you. 

I  think  Congressman  Gutknecht  left — oh,  no,  I  am  sorry.  There 
he  is  leaning  back. 

Mr.  Gutknecht.  I  am  here,  but  I  would  yield  to  somebody  else 
who  has  questions. 

Mr.  Ehlers.  Congressman  Gutknecht  yields  to  Representative 
Morella  who  has  recently  joined  us. 

Mrs.  Morella? 

Mrs.  Morella.  I  thank  the  gentleman  for  yielding  to  me,  and  I 
thank  you,  Mr.  Chairman. 

With  the  budget  released  today,  what  is  NSF's  budget  for  the 
Supercomputer  Centers  for  Fiscal  Year  1997? 

Dr.  Young.  For  Fiscal  Year  1997  it  will  be  essentially  the  same 
as  Fiscal  Year  1996,  which  is  roughly — and  I  may  be  wrong  by  a 
little  here — $67  million. 

Mrs.  Morella.  Sort  of  level  funding 

Dr.  Young.  Level  funding. 

Mrs.  Morella.  (continuing)  — for  the  18  months.  And  I  guess 
that  ties  into  the  question  that  was  asked  of  the  four  directors  that 
will  be  succeeding  you  on  this  whole  concept  of  is  it  collaboration, 
or  is  it  competition?  And  does  that  then  set  out  turfdoms?  And 
what  do  you  do  about  that? 

Is  there  anything  further  you  would  like  to  add? 

Dr.  Young.  When  they  testify,  the  directors  themselves  may 
want  to  testify  about  that. 

I  think  one  of  the  big  successes  of  the  Supercomputer  Center 
Program  over  the  past  10  years  has  been  the  degree  to  which  com- 
peting programs  have  learned  to  work  in  a  cooperative  fashion. 
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I  think  there  has  been  a  great  deal  of  collaboration  and  very 
fruitful  interaction  among  the  existing  Supercomputer  Centers. 

I  think  one  model  of  the  developing  program  is  that  we  will  have 
partnerships  that  extend  beyond  the  current  leading  edge  sites.  To 
some  extent  that  is  modeled  on  cooperation  that  already  exists. 

It  is  unquestionably  true  that,  in  the  context  of  a  recompetition 
that,  it  does  encourage  each  of  the  current  centers  that  may  be 
coming  in  in  the  recompetition  to  put  forward  the  best  proposals. 

At  the  same  time,  it  is  NSF's  management's  intent  to  continue 
cooperation  among  leading  edge  sites  and  partnerships  as  they 
come  in. 

But  you  are  right.  It  presents  new  challenges  for  our  manage- 
ment, and  we  intend  to  follow  through  on  that,  and  we  intend  to 
monitor  cooperation  and  put  in  place  procedures  which  will  encour- 
age cooperation  in  the  future. 

Mrs.  MORELLA.  So  with  the  procedures  you  have  in  mind  and  the 
concept,  you  think  this  is  going  to  work  out  well. 

Dr.  Hayes,  do  you  want  to  add  to  that  in  any  way? 

Dr.  Hayes.  Yes.  I  would  add  two  perspectives  which  expand  on 
the  points  that  Paul  has  made. 

One  is  that  competition,  if  it  is  done  in  an  appropriate  environ- 
ment, can  bring  out  creative  new  ideas  and  it  has  many  of  the  posi- 
tive aspects.  So  the  challenge — and  I  think,  adding  to  Paul's 
point — the  challenge  for  NSF  is  to  figure  out  how  you  keep  the 
playing  field  level  and  permit  an  appropriate  level  of  competition 
to  bring  out  the  creative  juices,  if  you  will,  as  part  of  this  program. 

And  I  think  there  are  sort  of  two  levels  of  partnerships  that  they 
have  to  pay  attention  to. 

One  is  the  partnerships  within  the  partnership  centers.  Then, 
once  the  competition  is  over — and  the  Task  Force  was  very  strong 
in  recommending  that  two  centers  is  infinitely  better  than  one; 
okay?  so  our  expectation  is  that  when  there  are  two  or  more  win- 
ners, there  is  still  an  important  issue  of  partnerships  among  the 
centers  that  win  out. 

NSF  has  a  good  record  with  the  present  centers  in  encouraging 
and  facilitating  that  kind  of  collaboration  and  partnership,  as  well. 

Dr.  Young.  May  I  come  back  to  that 

Mr.  Ehlers.  Yes,  you  may. 

Dr.  Young,  (continuing)  — in  addressing  partly  that  question  and 
the  earlier  question  about  why  take  a  successful  program  and 
recompete  it,  which  I  think  was  implicit. 

The  current  program  has  been  very  successful,  but  part  of  NSF's 
standard  management  is  to  take  peer  review  of  the  very  best  ideas 
and  always  put  things  out  for  recompetition  after  a  certain  amount 
of  time.  We  think  that  is  very  important. 

We  think  that  with  the  current  solicitation  we  are  going  to  get 
a  lot  of  new  ideas,  and  very  creative  ideas,  on  how  to  structure  this 
program. 

Ed  mentioned  earlier  in  his  testimony  that  our  solicitation  allows 
for  a  great  deal  of  creativity  in  how  these  programs  come  together, 
and  I  think  a  little  competition  and  a  chance  for  people  to  come  in 
with  new  ideas  about  the  appropriate  structure  of  the  program  is 
going  to  lead  to  a  very  exciting  new  program,  and  I  think  that  is 
going  to  work  well. 
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Mrs.  MORELLA.  I  just  wondered,  if  I  have  another  moment,  sim- 
ply because  from  what  I  have  read — and  I  am  not  expert  on  this — 
that  there  have  been  high  marks  given  by  at  least  two  review  pan- 
els of  the  current  system  and  why  modifications  can't  be  made 
without  the  additional  expense  that  would  be  incurred  by  the 
recompetition. 

Do  you  have  any  comments?  Do  you  wonder,  too? 

Dr.  Young.  I  think  I  actually  tried  to  answer  that  in  the  earlier 
response.  I  believe  that  we  will  get  a  significant  restructuring  and 
better  use  of  existing  technologies  through  the  competition. 

There  is  a  price  to  doing  that,  but  I  think  one  of  the  things  that 
is  true  about  Foundation  policies  and  procedures  generally  is  that 
we  find  open  competition — peer-reviewed — a  useful  way  to  bring 
out  new  ideas,  and  that  is  a  process  we  are  going  through.  It  is 
often  painful  and  it  is  not  without  cost,  but  we  do  believe  that  that 
is  the  best  way  to  bring  new  ideas  forward,  and  I  think  this  new 
solicitation  will  do  that. 

Mrs.  MORELLA.  Thank  you.  Dr.  Young. 

Mr.  Ehlers.  The  gentlelad/s  time  has  expired,  but  if  you  have 
a  brief  comment  you  can  add  to  that. 

Dr.  Hayes.  Just  to  follow  up  on  the  competing  visions  for  the  fu- 
ture in  terms  of  how  one  is  going  to  deliver  the  highest  perform- 
ance computing,  I  think  it  is  important  to  recognize  that  there  have 
been  significant  developments  in  the  networking  capability,  and 
they  are  projected  for  the  future. 

People  have  different  and  legitimate  differences  of  opinion  about 
what  the  future  holds.  So  one  of  the  things  that  is  going  to  be 
worked  out  as  part  of  this  competition  is  just  what  the  balance  is 
between  how  people  are  going  to  be  doing  the  highest  end  comput- 
ing and  the  extent  to  which  they  will  be  doing  it  in  a  distributed 
mode. 

Mrs.  MoRELLA.  Thank  you,  very  much. 

Mr.  Ehlers.  We  have  been  joined  by  Congressman  Volkmer.  Do 
you  have  any  questions,  Mr.  Volkmer? 

Mr.  Volkmer.  Yes. 

I  want  to  first  apologize  for  not  being  here  for  the  full  testimony, 
but  I  reviewed  some  of  your  statements  and  I  was  just  discussing 
it  with  the  gentleman  from  Pennsylvania  and  I  would  like  to  con- 
tinue along  a  similar  line  because  I  just  do  not  quite  understand 
the  reasoning  for  the  recompetition.  But  from  what  he  has  told  me, 
perhaps  there  is  a  need  for  it  if  you  want  to  try  something  dif- 
ferent. 

But  do  you,  when  you  envision  what  is  going  to  come  from  the 
new  competition,  do  you  have  any  ideas  of  what  you  are  looking  for 
to  come  forward? 

Dr.  Young.  Yes. 

Mr.  Volkmer.  What  are  those  ideas? 

Dr.  Young.  We  believe  that  networking  technology  and  the  ad- 
vent of  scalable  mid-level  parallel  machines  that  scale  up  to  the 
high  end  enable  us  to  restructure  a  program  in  which  we  can  get 
a  broader  distribution  of  the  technology  across  the  country,  more 
participating  centers,  and  better  integration  of  mid-level  systems 
and  better  use  of  high-speed  networking  connections  into  a  pro- 
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gram  that  is  less  centralized  and  broader  than  the  current  pro- 
gram. 

Mr.  VOLKMER.  All  right.  So  what  you  envision  is,  you  would  still 
have  two,  three,  four,  or  five  main  centers? 

Is  that  correct? 

Dr.  Young.  That  is  correct. 

Mr.  VoLKMER.  But  they  would  be  friendly  users? 

Dr.  Young.  Yes. 

Mr.  VOLKMER.  So  that  other  universities  and  other  research  cen- 
ters would  be  able  to  utilize  the  information  that  they  have  devel- 
oped, or  what? 

Dr.  Young.  Not  only  utilize  the  information  they  have  developed, 
but  partner  in  the  development  of  new  software  to  partner  in  the 
development  of  new  algorithms  and  work  in  a  fashion  that  is  coop- 
erative with  them  using  the  fact  that  they  have  similar  models  of 
machines  and  similar  software  developments. 

Ed  spoke  earlier  of  the  fact  that  the  expertise  in  computing  in 
the  country  is  not  located  at  just  a  few  institutions.  We  believe  that 
we  can  get  a  better  partnering  of  the  expertise  that  is  around  the 
country  by  a  program  that  makes  use  of  these  new  technologies. 

Mr.  VOLKMER.  And  the  ultimate  aim  within  four  to  five  to  six 
years  is  what? 

Dr.  Young.  The  ultimate  aim  is  to  have  the  ability  to  do  com- 
putational science  and  engineering  distributed  even  more  broadly 
across  the  country  and  have  a  more  powerful  and  better  program 
for  the  country  as  a  whole. 

Mr.  VOLKMER.  Would  we  end  up  then  with  a  perhaps  new  and 
different  software 

Dr.  Young.  We  will  certainly  have  new  and  better  software,  and 
we  think  that  the  restructured  program  will  actually  aid  in  that  de- 
velopment of  software  both  for  the  underlying  technology  and  for 
the  specific  technology  for  computational  science  and  engineering. 

Mr.  VOLKMER.  And  this  could  be  transferred  to  private  industry 
and  private  business? 

Dr.  Young.  We  believe  that  much  of  the  work  that  goes  on  in  the 
Foundation  in  this  area  ultimately  finds  application  in  business,  for 
sure.  The 

Mr.  VOLKMER.  All  right.  Fine. 

Now  do  you  envision  that  those  who  will  be  entering  the  competi- 
tion— or  is  the  competition  still  open? 

Dr.  Young.  Pardon? 

Mr.  VoLKMER.  Is  the  recompetition  still  open? 

Dr.  Young.  Yes,  it  is.  The  deadline  for  preproposals  is  April 
15th. 

Mr.  VOLKMER.  All  right. 

Do  you  envision — about  how  many  will  be  in  that  competition? 

Dr.  Young.  I  would  guess  on  the  order  of  four  to  half  a  dozen, 
maybe  more. 

Mr.  VOLKMER.  Four,  five,  six,  seven? 

Dr.  Young.  Yes. 

Mr.  VOLKMER.  Mostly  the  same  centers  that  are  out  there  right 
now? 

Dr.  Young.  That  is  the  current  indication,  I  believe. 
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Mr.  VoLKMER.  So  basically  we  have  a  recompetition  with,  to  me, 
a  goal  of  changing  what  is  being  done  right  now  at  those  same  cen- 
ters, it  looks  like. 

Dr.  Young.  It  will  change  what  is  going  on  at  those  centers,  and 
it  will  change  what  is  going  on  in  other  centers  around  the  country. 

Mr.  VOLKMER.  Well  the  other  centers  will  change  as  a  result  of 
what  these  centers  do.  Is  that  correct? 

Dr.  Young.  I  think  both  will  change. 

Ed? 

Dr.  Hayes.  I  think  I  need  to  draw  a  little  cartoon  of  what  this 
Partnership  Center  looks  like. 

Mr.  VoLKMER.  All  right.  That  might  help. 

Dr.  Hayes.  A  Partnership  Center  has  a  leading  edge  site 

Mr.  VOLKMER.  All  right. 

Dr.  Hayes,  (continuing)  — which  has  a  high-end  balanced  system. 
But  as  part  of  the  partnership  there  may  be  four  to  five — the  exact 
numbers  will  depend  on  the  proposals  of  course — but  of  partner- 
ships where  there  is  human  resource  capability;  there  are  existing 
networks;  and  there  will  be  hardware  and  software  capability. 

The  center  is  the  combination  of  the  leading  edge  sites,  as  well 
as  the  partnership  sites. 

Mr.  VoLKMER.  In  other  words 

Dr.  Hayes.  It  has  been  recognized  since  the  beginning  of  high- 
performance  computing  centers  that  a  good  deal  of  the  use  of  these 
high-end  systems  in  fact  is  used  by  people  who  have  major  research 
problems,  but  on  a  job-by-job  basis,  particularly  in  the  development 
of  the  software,  may  not  require  access  to  the  highest  end  of  the 
performance  spectrum. 

So  within  this  new  Partnership  Center  there  will  be  the  possibil- 
ity, particularly  in  the  early  development  stages,  to  allocate  time, 
resources,  and  training  on  the  mid-range  machine. 

Why? 

Because  now  it  is  technically  possible  today  to — for  instance  Ohio 
State  is  affiliated  with  the  Ohio  Supercomputing  Center,  which 
serves  all  the  universities  within  Ohio,  which  is  a  Metacenter  Re- 
gional Affiliate,  which  is  affiliated  with  the  Pittsburgh  Center,  but 
it  is  very  loosely  coupled. 

It  is  not  possible  for  the  Allocations  Committee  to  say  yes  to  a 
researcher,  we  think  you  have  a  good  idea  but  we  want  more  quan- 
titative data.  We  want  you  to  go  to  a  Partnership  Center  like  the 
Ohio  Supercomputing  Center  and  demonstrate  to  us  that  you  are 
technically  ready  to  make  effective  use  of  the  high-end  resource. 

When  you  are  ready,  then  you  will  be  able  to  compete  with  oth- 
ers to  have  that  access. 

I  think  in  a  sense  it  is  akin  to  getting  preliminary  data  for  a 
large  experiment.  This  is  viewed  as  a  more  cost-effective  method  of 
doing  that  because  now  it  is  the  whole  center  that  is  being  man- 
aged. 

The  other  point  has  to  do  with  software  productivity.  I  think  es- 
sentially everyone  who  has  been  looking  at  the  performance  of  the 
centers  program  over  the  years  has  recognized  the  important 
science  that  has  been  done.  But  the  advances  in  terms  of  software 
productivity  have  not  been  in  parallel — have  not  made  progress  in 
parallel  with  the  underlying  technology  that  is  being  used. 
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One  of  the  hopes  is  that  by  bringing  in  a  broader  set  of  partners, 
computer  scientists,  mathematicians,  that  can  work  on  early  ver- 
sions of  machines  which  may  at  some  point  be  the  leading  edge 
systems,  that  we  will  get  significant  advances  upon  which  everyone 
will  benefit  in  terms  of  software  productivity  and,  importantly,  in 
learning  how  to  use  distributed  systems  over  the  network. 

I  think  the  issue  is  on  what  sort  of  time  scale  are  we  going  to 
find  that  the  highest  performance  computers  in  fact  are  assembled 
by  using  computers  that  are  distributed  across  the  network. 

This  is  a  very  important  issue  for  industry,  as  well  as  for  re- 
search in  this  country. 

Mr.  VOLKMER.  Mr.  Chairman 

Mr.  Ehlers.  The  gentleman's  time  has  expired. 

Mr.  VOLKMER.  May  I  ask  one  more  question? 

Mr.  Ehlers.  If  it  is  a  very,  very  brief  one. 

Mr.  VOLKMER.  It  is. 

You  have  the  primary  center,  what  I  call  the  primary  center,  and 
an  ancillary  leading  edge. 

Dr.  Hayes.  Yes. 

Mr.  VoLKMER.  Now  those  that  join  with  that  leading  center, 
would  that  be  only  university  and  academia?  Or  will  that  include 
industry  and  private  business  at  the  research  centers? 

Dr.  Hayes.  We  include  industry,  national  laboratories-I  think 
there 

Mr.  VOLKMER.  Whoever  they- 


Dr.  Hayes.  It  is  specifically  stated  in  the  program 

Mr.  VOLKMER.  Whoever  they  can  get  to  sign  up,  in  other  words? 

Dr.  Young.  That  is  right.  I  could  read  from  the  program  solicita- 
tion. You  probably  do  not  want  that,  but  just  a  subset  of  those  list- 
ed. It  is  not  an  exclusive  list,  but  it  includes  universities,  and  in- 
cluding research  groups  within  universities,  centers  of  various 
forms,  research  and  educational  consortia  organizations  and 
groups,  regional  and  state  supported  high  performance  computing 
centers,  private-sector  organizations,  and  national  labs. 

That  list  is  not  intended  to  be  exclusive. 

Mr.  VOLKMER.  Thank  you. 

Mr.  Ehlers.  Thank  you. 

We  have  been  joined  by  Congressman  Boehlert.  Congressman,  I 
would  like  to  ask  you  to  take  the  Chair,  and  then  you  may  have 
your  turn  to  ask  questions,  as  well,  because  I  will  have 

Mr.  Boehlert.  What  a  deal  that  is. 

[Laughter.] 

Mr.  Ehlers.  Because  I  have  to  leave  for  the  Floor  to  defend  a 
bill. 

Mr.  Boehlert  [presiding].  What  a  deal  this  is.  I  get  the  Chair 
and  the  opportunity  to  question. 

I  am  not  quite  certain  I  am  clear  in  my  own  mind  why  we  are 
going  in  the  direction  we  are  going,  because  what  I  see  is  ten  years 
of  success  and  we  have  got  four  outstanding  centers  doing  a  lot  of 
things  you  are  talking  about — their  extensive  outreach  program, 
their  partnering,  bringing  in  the  private  sector — I  mean,  you  do  not 
have  just  four  little  islands  unto  themselves  scattered  around  the 
country. 
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It  seems  to  me  you  are  talking  about  reducing  the  number  of  cen- 
ters at  a  time  when  I  would  think  we  would  be  expanding  what  we 
are  doing  at  the  centers  and  encouraging  them  to  keep  up  the  good 
work. 

I  am  not  quite  sure  I  am  clear  why  there  seems  to  be  the  drive 
toward  a  reduction  in  the  number  of  centers  after  a  decade  of  suc- 
cess. 

And  once  again,  I  seem  to  think — or  that  is  my  impression  at 
least — that  the  centers  are  all  individually  doing  exactly  what  you 
are  talking  about  what  you  envision  as  a  result  of  this  competition, 
the  extensive  outreach;  the  partnering;  the  bringing  in  the  private 
sector. 

Enlighten  me  a  little  bit,  if  you  will.  Doctor? 

Dr.  Young.  Shall  I  try  that  first  and  then  let  you  come? 

Dr.  Hayes.  Yes. 

Dr.  Young.  I  think  your  point  is  well  taken.  It  may  be  a  case 
of  success  breeding  success. 

But  I  think  it  is  also  true  that  the  technologies  permit  an  accel- 
eration of  that  process  and  a  deepening  of  that  process. 

So  I  think  that  you  will  see  more  partnering  and  more  outreach 
with  the  program  as  it  is  structured. 

As  Ed  mentioned  earlier,  and  I  think  before  you  came  in,  the 
structure  that  is  envisioned  here  is  a  structure  that  makes  sense 
independently  of  the  actual  number  of  centers  and  partnerships.  So 
we  think  that  this  intent  to  make  better  use  of  networking  and 
scalable  parallel  computing  is  something  that  the  centers,  as  you 
point  out,  have  been  doing,  and  we  think  it  is  a  natural  evolution 
of  the  program  in  that  direction. 

At  the  same  time,  we  do  have  constant  budgets,  tight  budgets, 
and  we  still  have  inflation.  There  is  a  limited  amount  we  can  do 
with  tight  budgets. 

Ed  mentioned  that  one  of  the  things  we  need  to  do  at  the  leading 
edge  sites  is  to  keep  them  one  or  two  orders  of  magnitude,  ten  to 
one  hundred  times  faster,  ten  to  one  hundred  times  more  storage 
power,  just  the  technology  needs  to  be  a  couple  of  places  ahead. 
And  that  is  expensive  and  there  are  a  limited  number  of  places  we 
can  do  that. 

We  think  we  are  going  to  get  a  more  efficient  program  as  we  ex- 
pand some  of  the  things  the  centers  have  been  doing  very  well. 

Mr.  BOEHLERT.  Dr.  Hayes,  do  you  wish  to  add  anything  to  that? 

Dr.  Hayes.  Yes.  I  think  one  of  the  futures  that  the  Task  Force 
was  very  concerned  about  is  the  scenario  where  we  get  four  Part- 
nership Centers,  where  the  Metacenter  Regional  Affiliates  that  we 
currently  have,  that  that  part  of  the  program  is  folded  into  the 
broader  program,  so  you  get  four  Partnership  Centers. 

Mr.  Boehlert.  Which  is  in  effect  what  we  have  right  now. 

Dr.  Hayes.  Well  I  will  come  back  to  that  as  a  detail.  That  is  not 
quite  right.  We  have  four  leading  edge  sites  right  now,  and  we 
have  loosely  coupled  Metacenter  Regional  Affiliates.  That  is  what 
we  presently  have. 

The  concern  is  that,  with  the  changing  philosophy  at  ARPA, 
DARPA  now,  that  if  these  centers  came  into  being  and  the  NSF 
budget  did  not  grow  at  a  rate  that  was  significantly  above  inflation 
for  this  program,  that  you  would  not  be  able  to  keep  up  with  the 
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recapitalization  cycle  that  would  be  necessary  to  keep  the  leading 
edge  sites  at  a  level  that  would  be  sufficiently  interesting  to  draw 
the  very  best  researchers  to  the  centers. 

You  may  want  to  explore  this  with  the  center  directors,  but  the 
impression  that  many  of  us  have  is  that  without  the  DARPA 
money  to  help  NSF  move  into  the  massively  parallel  arena  over  the 
last  five  years  of  the  program,  why  many  of  the  centers  would  have 
been  stuck  at  the  vector  computing  stage,  because  many  had  long- 
term  leases  or  payments  on  existing  centers.  And  without  an  infu- 
sion or  a  recapitalization,  you  could  find  that  what  appeared  to  be 
a  good  investment  and  positioned  the  center  for  a  period  of  two  or 
three  years  basically  put  a  Partnership  Center  in  the  position  that 
they  were  not  really  capable  of  supporting  the  most  interesting 
science,  and  that  that  would  basically  put  you  on  the  kind  of  spiral 
where  the  scientific  community,  the  people  who  actually  do  the 
science,  would  say,  well,  by  buying  a  mid-range  system  at  signifi- 
cantly lower  cost,  they  can  do  research  which  is  competitive  with 
what  you  could  do  at  a  Partnership  Center  with  six-  or  seven-year- 
old  technology. 

So  the  concern  about  "sizing"  the  program  that  was  very  much 
on  our  mind  is  that  the  program  needs  to  be  sized  so  that  it  can 
be  recapitalized  on  a  reasonable  time  scale. 

People  debate  about  whether  that  is  three  or  four  years,  but  I 
think  there  are  a  few  that  would  argue  that  five  to  seven  years  is 
the  right  recapitalization  schedule. 

So  when  you  think  about  within  the  kind  of  budget  environment 
they  are  in,  and  the  kind  of  funding  that  is  available  for  the  Cen- 
ters Program,  recapitalization  on  a  shorter  time  scale,  and  also 
bringing  in  the  partnership  aspect  and  being  able  to  do  realistic 
and  interesting  experiments  with  distributed  computing,  the 
present  centers  do  not  have  that  to  the  same  extent  that  is  envi- 
sioned in  our  recommendations. 

The  current  centers  are  doing  many  very  important  things,  and 
I  do  not  want  to  in  any  way  be  critical  of  what  they  are  doing 
under  the  present  program — and  I  think  the  Task  Force  wasn't. 

But  underlying  all  of  this  is  the  fact  that,  you  know,  we  are 
Blessed  with  the  fact  that  technology  is  changing  very  rapidly,  and 
in  thinking  about  what  the  program  may  look  like  over  a  five-  to 
ten-year  period  we  need  to  have  the  flexibility  within  the  program 
so  that  it  can  recapitalize  and  keep  up  with  the  technology. 

Mr.  BOEHLERT.  It  seems  to  me  you  are  making  a  persuasive  case, 
or  fighting  like  hell  for  more  bucks  for  the  program,  rather  than 
downsizing  the  program  or  restructuring  the  program. 

Is  it  your  interpretation  of  budget  realities  that  is  forcing  this 
new  line  of  thinking? 

Dr.  Hayes.  Let  me  be  clear  about  two  points. 

There  is  nothing  about  the  partnership  concept  that  is  related  to 
the  budget.  I  think  the  Task  Force  would  be  unanimous  that  the 
partnership  aspect,  where  you  have  a  leading  edge  site  that  is 
tightly  coupled,  not  loosely  coupled  but  tightly  coupled  to  partner- 
ship sites,  is  the  right  model  for  the  future. 

Mr.  BoEHLERT.  But  is  that  not  what  is  evolving  right  now  under 
the  present  configuration? 
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Dr.  Hayes.  There — I  would — you  will  have  to  ask  the  center  di- 
rectors their  view  on  that.  I  happen  to  be  familiar  with  one 
Metacenter  Regional  Affiliate.  There  are  a  number  of  aspects  that 
we  would  expect  to  be  present  in  a  Partnership  Center  in  terms  of 
tight  coupling  and  management  and  planning,  which  at  least  do 
not  appear  to  be  present  in  that  particular  Metacenter  Regional  Af- 
filiate. 

The  coupling  is  much  looser.  The  planning  is  looser.  And  the 
ability  to — in  fact  there  is  no  ability  for  an  Allocations  Committee 
to  allocate  resources  at  the  Metacenter  Regional  Affiliate  that  I  am 
familiar  with. 

So  if  a  proposal  comes  into  NSF  from  New  York  State,  for  in- 
stance, and  says,  look,  they  want  to  carry  out  a  particular  project, 
and  the  Allocations  Committee  says,  well,  what  you  need  is  a  mid- 
range  kind  of  machine,  and  that  machine  is  available  at  the  Ohio 
Supercomputing  Center,  there  is  no  way  that  they  can  allocate 
time  on  that  particular  resource. 

So  what  happens  is,  either  the  particular  project  does  not  get  any 
allocation  at  all  because  there  is  uncertainty  about  whether  it  is 
technically  ready,  you  see,  or  the  decision  is  made,  sometimes  pre- 
maturely, to  allocate  resources  on  a  very  high-end  system  that  is 
costly  to  iDasically  demonstrate  that  they  are  technically  ready. 

It  seems  to  me  that  leads  to  inefficiencies  in  the  program. 

Mr.  BOEHLERT.  Congressman  Doyle? 

Mr.  Doyle.  Thank  you. 

I  just  want  to  get  one  thing — ^We  have  four  centers  right  now. 
And,  Mr.  Boehlert,  when  you  were  talking  about  what  is  going  on, 
I  think  the  reality  is  here,  though,  that  under  what  is  being  pro- 
posed here,  what  we  are  going  to  end  up  with  is  one,  maybe  two 
leading  edge  centers. 

Is  anybody  disputing  that? 

Do  you  see  more  than  two  leading  edge  centers,  given  the  budget 
constraints  we  are  under  right  now? 

Dr.  Young.  I  think  it  is  plausible  that  we  will  end  up  with  be- 
tween one  and  three. 

Mr.  Doyle.  Three  now? 

Mr.  Brown.  Between  one  and  three. 

[Laughter.] 

Mr.  Doyle.  Yes.  "Between  one  and  three"  is  two,  yes. 

[Laughter.] 

Mr.  Doyle.  And  the  other  observation  is,  I  am  just  sitting  here 
looking  and  thinking  that  a  lot  of  the  changes  you  are  talking 
about,  while  they  are  going  to  open  up,  you  know,  more  to  mid- 
range  users,  it  seems  to  me  it  is  going  to  be  at  the  expense  of  the 
high-end  users;  and  the  high-  end  users  are  oversubscribed  as  we 
speak. 

I  am  just  wondering,  you  know,  it  gets  back  to  what  Mr.  Boehlert 
was  sajdng,  too.  Maybe  we  need  some  people  to  sit  up  here  and  talk 
about  the  need  for  some  additional  funding  for  what  has  been  a 
very  successful  program  at  these  four  regional  centers. 

It  just  seems  that  what  is  going  to  happen  here  is  we  are  going 
to  open  up  the  process  for  some  at  the  expense  of  high-end  users 
that  are  already  over-subscribed  and  end  up  with  one,  maybe  two. 
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leading  edge  facilities  here  instead  of  our  four  regional  centers. 
That  is  just  what  it  appears  to  me  to  be,  anyway. 

I  would  be  interested  to  hear  your  comments. 

Dr.  Hayes.  I  think  the  point  I  made  in  my  prepared  statement 
about  the  process  that  NSF  has  is  that  I  think  the  Task  Force  felt 
comfortable  with  the  concept  of  the  Partnership  Centers,  as  I  said. 

But  the  subsequent  steps  that  took  place  within  the  Foundation 
were  that  basically  the  Director  had  to  take  a  plan  to  the  National 
Science  Board,  and  the  Director  and  the  National  Science  Board  in 
the  context  of  the  whole  NSF  program,  the  whole  budget,  had  to 
make  a  judgment  on  this  as  to  what  was  a  reasonable  balance  be- 
tween high-performance  computing  and  the  rest  of  the  NSF  pro- 
gram. 

The  Task  Force  certainly  heard  loud  and  clear  that  many  of  the 
people  that  are  using  the  existing  centers  in  fact  depend  upon  sup- 
port from  NSF  through  disciplinary  programs  and  other  programs 
for  their  support. 

So  if  you,  again,  look  at  things  in  that  kind  of  perspective,  some- 
one has  to  look  at  the  total  NSF  portfolio  within  what  is  a  reason- 
able expectation  of  the  budget.  I  think  this  Committee  looking  at 
this  is  very  important  can  make  a  judgment  that  we  want  to  keep 
the  whole  organic  whole  healthy. 

So  if  you  have  somebody  like  Jerry  Ostriker,  who  Paul  men- 
tioned, or  another  one  of  the  major  users  basically  that  loses  fund- 
ing for  their  overall  program  which  is  not,  you  know,  exclusively 
a  high-  performance  computation  program,  if  you  visit  any  one  of 
these  laboratories,  including  Ostriker's,  he  has  a  lot  of  mid-range 
computing  capability  in  his  laboratory  that  is  coupled  appro- 
priately. 

Not  everyone  is  so  fortunate.  I  use  Jerry  Ostriker  as  an  example 
because  I  think  it  is  quite  unlikely  that  in  the  budget  reduction  he 
loses  his  support,  but  I  think  that  that  is  one  of  the  issues  as  to 
what  gives  us  the  best  overall  plan. 

The  sense  of  the  Task  Force  was  that  you  really  need  to  have  a 
minimum  of  two  Partnership  Centers.  And  if  the  budget  can  afford 
additional  centers  or,  importantly,  if  there  is  additional  significant 
cost  sharing  that  comes  from  vendors  and  from  the  partners  and 
the  leading  edge  sites  and  from  the  states,  that  in  fact  it  may  be 
possible  to  do  more  than  two.  And  if  the  NSF  budget  would  sup- 
port, with  the  recapitalization  that  I  mentioned  earlier,  more  than 
the  minimum  of  two  that  we  were  strongly  pushing  for,  then  within 
the  concept  of  the  partnership  I  think  there  will  be  quite  a  comfort 
level  and  an  enthusiasm  for  doing  that.  But  only  if  it  meets  the  re- 
capitalization and  the  balance  aspects  that  I  was  talking  about. 

Mr.  BOEHLERT.  Thank  you  very  much,  and  thank  both  of  you 
very  much. 

We  will  now  move  to  the  next  panel  consisting  of  Dr.  Malvin 
Kalos,  Director  of  the  Cornell  Theory  Center;  Dr.  Larry  Smarr,  Di- 
rector of  the  National  Center  for  Supercomputing  Applications;  Dr. 
Douglas  Pewitt,  Acting  Director,  San  Diego  Supercomputing  Cen- 
ter; and  Dr.  Ralph  Roskies,  a  Scientific  Director  for  the  Pittsburgh 
Supercomputing  Center. 

Gentlemen,  your  statements  will  appear  in  the  record  in  their  en- 
tirety. I  would  ask  in  the  interests  of  time  you  give  us  as  much  as 
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we  need  to  know  so  that  we  can  ask  intelligent  questions  and  un- 
derstand the  issue,  but  please  do  not  be  too  lengthy. 

I  would  ask  also  that  we  follow  the  order  of  introduction  with  the 
privilege  of  leading  off  going  to  a  fellow  New  Yorker,  Dr.  Kalos,  as 
soon  as  you  are  comfortably  seated. 

Dr.  Kalos.  Thank  you 

Mr.  BOEHLERT.  And  would  you  pull  the  mike  a  little  bit  closer? 

STATEMENT  OF  DR.  MALVIN  H.  KALOS,  DIRECTOR,  CORNELL 
THEORY  CENTER,  AND  PROFESSOR  OF  PHYSICS,  CORNELL 
UNIVERSITY,  ITHACA,  NEW  YORK 

Dr.  Kalos.  Mr.  Chairman,  it  is  my  privilege  to  be  invited  to  com- 
ment on  the  NSF's  new  program  of  Partnerships  for  Advanced 
Computational  Infrastructure. 

I  am  Malvin  Kalos,  the  Director  of  the  Cornell  Theory  Center, 
one  of  the  four  current  NSF  Supercomputer  Centers.  I  am  a  Profes- 
sor of  Physics  at  Cornell  University.  I  have  been  a  computational 
scientist  for  more  than  40  years,  and  at  present  am  the  Vice  Chair- 
man of  the  Division  of  Computational  Physics  of  the  American 
Physical  Society. 

Our  Center  was  founded  with  the  premise  that  computation  has 
a  profound  effect  upon  all  the  sciences  and  engineering. 

From  the  beginning,  our  Center  took  a  leadership  role  in  scalably 
parallel  computing.  Now  computational  science  is  undergoing  a  rev- 
olution with  the  advent  of  practical  scalable  computing. 

The  512-processor  SP  at  Cornell  is  one  of  the  most  powerful  com- 
puting environments  available  today.  We  have  seen  leading  re- 
searchers in  our  community  cross  new  thresholds  of  capability  to 
do  qualitatively  new  science. 

In  the  end,  a  program  like  this  and  our  Center  can  only  be 
judged  by  the  quantity  of  important  scientific  knowledge  and  un- 
derstanding which  could  not  have  been  accomplished  without  it. 

I  would  like  to  call  attention  to  a  few  special  examples  that  illus- 
trate the  scope  and  influence  of  this  program. 

My  first  example  is  one  referred  to  by  Dr.  Young  in  the  impor- 
tance of  computation  for  molecular  biology: 

An  international  team  of  scientists  investigated  the  rapid  activity 
of  acetylcholinesterase  which  breaks  down  the  neurotransmitter  ac- 
etylcholine. 

Its  three-dimensional  crystal  structure  was  solved  a  few  years 
ago.  It  seemed  inconsistent  with  the  rapid  djmamics  of  the  enzyme. 

Using  computing  resources  at  all  four  of  the  NSF  Centers,  the  re- 
searchers discovered  that  there  was  a  back  door  to  the  molecule 
that  explains  the  rate. 

Since  inhibitors  of  acetylcholinesterase  are  important  medica- 
tions for  diseases  such  as  myasthenia  gravis,  glaucoma,  and  Alz- 
heimer's disease,  this  new  insight  may  lead  to  more  effective  drugs 
to  fight  these  diseases. 

A  second  example  is  in  astrophysics: 

Using  both  the  NSF-supported  Arecibo  radiotelescope  for  obser- 
vation and  the  Cornell  Theory  Center's  computational  facility  for 
intensive  analysis  and  modeling,  Alexander  Wolszczan  of  Perm 
State  University  succeeded  in  identifying  the  first  planets  outside 
of  our  solar  system.  Wolszczan  was  recently  awarded  the  pres- 
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tigious  Beatrice  M.  Tinsley  Prize  of  the  American  Astronomical  So- 
ciety for  this  discovery. 

John  Dawson  and  his  research  group  at  the  University  of  Cahfor- 
nia  at  Los  Angeles  studies  turbulence  in  complex  plasmas.  The 
power  of  the  Cornell  SP  permit  his  group  to  support  simulations 
that  are  an  order  of  magnitude  improvement  over  previous  ma- 
chines in  spatial  resolution,  which  has  allowed  them  for  the  first 
time  to  simulate  successfully  the  parameters  of  a  large  fusion  de- 
vice. 

Here,  as  elsewhere,  computation  as  an  aid  to  understanding  and 
as  an  aid  to  predesign  of  costly  experiments  is  an  important  way 
to  use  limited  budgets  in  an  optimum  way. 

At  the  Massachusetts  Institute  of  technology,  Edmund 
Bertschinger  conducts  Grand  Challenge  research  aimed  at  under- 
standing the  formation  of  galaxies  in  the  universe.  The  power  and 
memory  of  computers  such  as  the  SP  permit  him  and  the  rest  of 
the  Grand  Challenge  Team  to  study  models  large  enough  to  cap- 
ture these  large-scale  dynamical  effects.  This,  too,  is  work  that 
spans  the  NSF  Centers  Program. 

I  would  like  to  echo  Dr.  Hayes'  comments  about  the  relationship 
with  industry. 

We  believe  that  we  have  also  made  a  singularly  important  con- 
tribution to  the  national  scientific  effort  by  the  depth  and  quality 
of  our  partnership  with  IBM. 

Through  this  partnership,  we  rekindled  IBM's  interest  in  high- 
end  technical  computing  and  thereby  helped  to  bring  their  superb 
technology  to  the  service  of  science  and  engineering. 

As  is  often  the  case,  the  technical  computing  world  has  led  the 
way  in  bringing  out  new  technologies  for  the  broader  commercial 
arena. 

We  believe  that  IBM's  success  will  help  assure  a  future  for  com- 
putational science. 

Computation  is  a  unique  tool  that  permits  quantitative  connec- 
tions among  different  disciplines.  Every  one  of  the  large  problems 
that  confront  our  society,  and  to  whose  solutions  we  expect  science 
to  contribute,  is  in  face  a  multi-disciplinary  problem. 

Issues  of  the  environment  and  medicine,  to  cite  only  two,  involve 
many  sciences,  chemistry,  physics,  engineering,  fluid  flow,  biology, 
materials.  Bringing  the  knowledge  from  these  fields  together  to 
make  quantitative  predictions — for  example,  about  the  effects  of 
some  technological  proposal — would  be  utterly  impossible  without 
the  use  of  high-performance  computational  modeling. 

It  is  the  indispensable  natural  language  of  quantitative  multi- 
disciplinary  research. 

Computational  science  is  now  an  essential  tool  in  experimental 
science.  The  most  advanced  scientific  instruments — optical  and  ra- 
dial telescopes,  particle  accelerators,  and  computers  themselves — 
are  designed,  studied,  optimized,  and  verified  with  computer  sim- 
ulation. 

Data  collection  is  usually  automated  and  the  reduction  to  com- 
prehensible data  sets  or  conceptual  models  may  involve  enormous 
computations  as  in  the  case  of  Alexander  Wolszczan. 
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Now  the  entire  scientific  and  engineering  community  of  the  coun- 
try has  the  opportunity  to  exploit  the  new  tools  of  computational 
science. 

We  share  NSF's  vision  that  education  of  tomorrow's  young  sci- 
entists includes  bringing  them  into  the  process  of  exploiting  the 
Nation's  most  advanced  tools,  including  the  computational  re- 
sources of  the  NSF  centers.  They  will  carry  the  message  to  the  rest 
of  our  society  and  to  the  future. 

The  supercomputing  community  now  finds  itself  at  a  major  cross- 
roads. To  exploit  these  new  machines,  the  scalably  parallel  ma- 
chines, a  major  retooling  of  software  and  algorithms  must  take 
place. 

In  response  to  the  new  NSF  program,  but  also  to  assure  that  re- 
cent advances  are  sustained  and  extended,  and  that  new  challenges 
can  be  met.  The  Cornell  Center  is  creating  a  National  Alliance  for 
Advanced  Computational  Infrastructure.  This  will  comprise  a  new 
network  of  cooperating  organizations  around  the  Nation  that  will 
bring  their  varied  talents  and  their  resources  to  the  Alliance,  along 
with  the  world-class  supercomputing  infrastructure  of  the  Theory 
Center  and  its  strong  and  experienced  professional  staff. 

Our  partners  are  of  world-class  stature  in  scalable  computing,  in 
related  technologies  such  as  virtual  reality  and  computer  graphics, 
in  all  areas  of  computer  science  that  relate  to  parallel  computing, 
in  computational  mathematics,  in  forefront  applications. 

Mr.  Chairman,  you  have  asked  how  the  new  competition  has  af- 
fected our  Center.  It  has  certainly  diminished  the  strong  and  fruit- 
ful collaboration  with  the  others  that  went  by  the  rubric  of 
"Metacenter." 

It  has  created  a  sense  of  uncertainty  about  the  future  in  the 
minds  of  some  of  our  professional  staff.  Naturally  the  realization 
that  this  is  a  very  complex  and  serious  competition  with  very  high 
stakes  has  generated  an  intense  concentration  on  winning,  distract- 
ing us  somewhat  from  our  normal  mission. 

In  the  long  term,  I  am  sure  that  the  intercenter  cooperation  will 
re-establish  itself.  Nevertheless,  we  share  a  commitment  to  the 
idea  that  a  partnership  of  the  kind  that  we  envision  is  a  logical 
next  step  for  the  Cornell  Theory  Center.  It  will  transform  our  pro- 
gram into  something  quite  new,  something  that  will  serve  the  best 
interests  of  the  community  devoted  to  computational  science,  and 
the  broader  community  of  research  in  basic  science. 

[The  prepared  statement  of  Dr.  Kalos  follows:] 

Testimony  of  Malvin  H.  Kalos  on  the  NSF  Partnership  for  Advanced 
Computational  Infrastructure 

before  the 

Subcommittee  on  Basic  Research  of  the  Committee  on  Science 

March  19,  1996 

Mr.  Chairman,  it  is  a  privilege  to  be  invited  to  comment  on  the  NSFs  new  pro- 
gram of  Partnerships  for  Advanced  Computational  Infrastructure. 

I  am  Malvin  Kalos,  the  Director  of  the  Cornell  Theory  Center,  one  of  the  four  cur- 
rent NSF  supercomputer  centers,  and  a  Professor  of  Physics  at  Cornell  University. 
I  have  been  a  computational  scientist  for  more  than  forty  years,  and  at  present  am 
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the  Vice  Chairman  of  the  Division  of  Computational  Physics  of  the  American  Phys- 
ical Society. 

Our  Center  was  founded  with  the  premise  that  computation  can  have  a  profound 
impact  upon  all  the  sciences  and  engineering.  From  the  beginning,  the  Center  took 
a  leadership  role  in  scalably  parallel  computing  as  the  ultimate  source  of  the  most 
processing  power  and  memory.  We  understood  that  orders  of  magnitude  increases 
in  computational  power  could  only  come  from  harnessing  large  numbers  of  commod- 
ity microprocessors,  rather  than  smaller  numbers  of  specially  fabricated  computer 
chips.  Now  computational  science  is  undergoing  a  revolution  with  the  advent  of 
practical  scalable  computing.  Computing  resources  have  increased  by  three  orders 
of  magnitude  in  speecf  and  memory  during  the  past  decade.  The  512-processor  SP 
at  Cornell  is  one  of  the  most  powerful  computing  environments  available  today.  We 
have  seen  leading  researchers  in  our  community  cross  new  thresholds  of  capability 
to  do  qualitatively  new  science — that  is,  to  attack  new  problems  which  could  not  be 
considered  before  and  old  problems  in  ways  that  provide  new  insights. 

In  the  end,  a  program  like  this  and  our  Center  can  only  be  judged  by  the  quantity 
of  important  scientific  knowledge  and  understanding  which  could  not  have  been  ac- 
complished without  it  and  by  the  impact  of  this  new  knowledge  upon  our  society. 

An  international  team  of  scientists  collaborated  to  investigate  the  rapid  activity 
of  the  enzyme  acetylchoinesterase.  The  enzyme  breaks  down  the  neurotransmitter 
acetylcholine  diffused  across  nerve  cell  synaptic  gaps.  Its  three-dimensional  crystal 
structure  was  solved  several  years  ago,  and  showed  the  active  site  of  the  enzyme 
to  be  a  long,  narrow  channel — too  narrow  to  deal  rapidly  with  the  job  of  breaking 
down  acetylcholine.  Yet  it  is  known  that  acetylcholinesterase  acts  very  rapidly. 
Using  computing  resources  at  all  four  of  the  national  centers,  the  researchers  dis- 
covered that  there  was  a  "back  door"  to  the  active  site  of  the  enzyme  that  explained 
its  ability  to  work  so  quickly.  Since  inhibitors  of  acetylcholinesterase  are  important 
medications  for  diseases  such  as  myasthenia  gravis,  glaucoma,  and  Alzheimer's  dis- 
ease, this  new  insight  may  lead  to  more  effective  drugs  to  fight  these  diseases. 

Using  both  the  Arecibo  radiotelescope  for  observation  and  the  Cornell  Theory  Cen- 
ter's computational  facility  for  intensive  analysis  and  modeling,  Alexander 
Wolszczan  of  Penn  State  University  succeeded  in  identifying  the  first  planets  out- 
side of  our  solar  system.  Wolszczan  was  recently  awarded  the  prestigious  Beatrice 
M.  Tinsley  Prize  of  the  American  Astronomical  Society  for  his  discovery. 

John  Dawson  and  his  research  group  at  the  University  of  California  at  Los  Ange- 
les use  particle-in-cell  methods  to  study  turbulence  in  complex  plasmas.  The  mem- 
ory and  speed  of  the  Cornell  SP  permit  his  group  to  perform  simulations  that  are 
an  order  of  magnitude  improvement  over  previous  machines  in  spatial  resolution, 
and  so  allow  them  to  see  interesting  and  important  new  effects.  They  have  been 
able,  for  the  first  time,  to  simulate  successfully  the  parameters  of  major  Tokomaks, 
such  as  the  TFTR  at  Princeton,  which  is  the  largest  fusion  device  that  exists  in  the 
United  States. 

A  number  of  cardiovascular  and  psychiatric  diseases  are  currently  treated  with 
drugs  that  act  on  the  neurotransmitter  serotonin  and  its  receptors.  The  cellular  re- 
ceptor for  serotonin  is  a  gatekeeper  molecule  that  recognizes  and  binds  the  serotonin 
and  then  transmits  the  signal  to  the  cell  by  binding  to  a  special  class  of  trans- 
ducers— the  G-proteins.  Researchers  at  Mount  Sinai  Medical  Center  made  a  break- 
through in  modeling  the  serotonin  receptor  using  CTC  resources.  The  breakthrough 
came  from  modeling  the  structural  changes  that  occur  in  the  serotonin  receptor 
when  it  binds  to  a  ligand  and  to  the  G-protein,  causing  it  to  carry  out  its  function. 
This  research  shows  how  G-proteins  can  be  switched  on  by  structural  changes  in 
specific  regions  of  the  receptor  molecule  and  is  expected  to  lead  to  the  development 
of  more  effective  drugs  for  a  vast  range  of  diseases. 

Edmund  Bertschinger,  Massachusetts  Institute  of  Technology,  conducts  Grand 
Challenge  research  aimed  at  the  understanding  of  the  formation  of  galaxies  in  the 
universe.  The  memory  and  power  of  computers  such  as  the  SP  permit  him  and  oth- 
ers to  study  model  systems  large  enough  to  capture  these  large-scale  dynamical  ef- 
fects. 

George  Karniadakis,  Brown  University,  is  studying  turbulent  flow  through  direct 
numerical  simulation  in  order  to  resolve  flow  fields  at  all  scales  from  first  principles. 
A  fundamental  goal  is  to  provide  insight  into  a  number  of  open  questions  in  fluid 
dynamics,  including  large-scale  instabilities  of  turbulent  wakes  and  associated  drag 
and  lift  variation  in  flows  past  bluff  bodies;  symmetry-breaking  bifurcations  and 
low-dimensionality  in  flow-structure  interactions;  and  shear  stress  control  and  vor- 
ticity  transport  in  wall-bounded  flows  using  both  passive  and  active  means  of  con- 
trol such  as  surface  modifications  and  electromagnetic  techniques. 

Toichiro  Kinoshita,  Cornell  University,  studies  the  anomalous  magnetic  moment 
of  light  elementary  electrons,  whose  interactions  are  primarily  electromagnetic  and 
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which  therefore  provide  very  stringent  tests  of  quantum  electrodynamics.  A  power 
series  expansion  of  the  magnetic  moment  gives  successive  terms  that  are  integrals 
over  more  and  more  dimensions.  They  are  done  by  a  very  sophisticated  Monte  Carlo 
but  require  much  more  resources  at  higher  orders.  Moving  to  the  SP  has  permitted 
Kinoshita  to  go  to  third  and  fourth  order. 

We  believe  that  we  have  also  made  a  singularly  important  contribution  to  the  na- 
tional scientific  effort  by  the  depth  and  quality  of  our  partnership  with  IBM. 
Through  this  partnership,  we  rekindled  IBM's  interest  in  high-end  technical  com- 
puting and  thereby  helped  to  bring  their  superb  technology  to  the  service  of  science 
and  engineering.  As  is  often  the  case,  the  technical  computing  world  has  led  the  way 
in  bringing  about  new  technologies  for  the  broader  commercial  arena;  we  believe 
that  IBM's  success  will  help  assure  a  future  for  computational  science. 

The  essence  of  pure  science  is  to  connect  our  scientific  knowledge  in  a  seamless 
web  of  quantitative  understanding.  This  has  become  harder  as  we  try  to  probe  into 
more  and  more  complex  phenomena  that  cannot  be  analyzed  by  the  mathematical 
tools  at  our  disposal.  Computational  modeling  is  essential  to  fill  this  need. 

Many  areas  of  science  involve  this  kind  of  systematic  connection  among  different 
phenomena  at  different  scales  of  length  or  energy.  All  aspects  of  chemistry,  biology 
and  medicine,  the  physics  of  materials,  astrophysics  and  many  others  require  this 
approach. 

Computational  science  is  now  also  an  essential  tool  in  experimental  science.  The 
most  advanced  scientific  instruments,  optical  and  radio  telescopes,  particle  accelera- 
tors, and  computers  themselves  are  studied,  designed,  optimized,  and  verified  with 
computer  simulation.  Data  collection  is  usually  automated,  and  the  reduction  to 
comprehensible  data  sets  may  involve  enormous  computations.  Exchange  of  large 
data  sets  will  require  very  heavy  use  of  future  high  capacity  data  networks. 

Now  the  entire  scientific  and  engineering  community  of  the  country  has  the  op- 
portunity to  exploit  the  new  tools  of  computational  science.  Students  and  young  sci- 
entists, who  are  always  the  very  heart  of  any  important  scientific  change,  are  in- 
volved. We  share  NSFs  vision  that  education  of  tomorrow's  young  scientists  includes 
bringing  them  into  the  process  of  exploiting  the  nation's  most  advanced  tools,  in- 
cluding the  computational  resources  of  the  National  Centers.  They  will  carry  the 
message  to  the  rest  of  our  society  and  to  the  future. 

Another  vital  role  of  computational  science  is  that  of  permitting  quantitative  con- 
nections among  different  disciplines,  that  is,  in  supporting  multidisciplinary  re- 
search. Every  one  of  the  large  problems  that  confront  our  society,  and  to  whose  solu- 
tions we  expect  science  to  contribute,  is  in  some  sense  a  multidisciplinary  problem. 
Issues  of  the  environment  and  medicine,  to  cite  only  two,  involve  many  sciences — 
chemistry,  physics,  engineering,  fluid  flow,  biology,  and  materials. 

Bringing  the  knowledge  from  these  fields  together  to  make  quantitative  pre- 
dictions about  the  effects  of  some  technological  or  regulatory  proposal  would  be  ut- 
terly impossible  without  the  use  of  high-performance  computational  modeling,  the 
indispensable  natural  language  of  quantitative  multidisciplinary  research. 

The  supercomputing  community  now  finds  itself  at  a  major  crossroads.  To  exploit 
these  new  machines,  a  major  re-tooling  of  software  and  algorithms  will  have  to  take 
place. 

Science  and  its  application  to  societal  problems  involve  the  national  community. 
Bringing  to  bear  the  transformation  made  possible  by  computational  science  in  the 
most  complete  and  positive  way  requires  that  its  techniques  and  strategies  be 
learned,  used,  and  shared  by  the  widest  possible  group  of  researchers  and  educators. 
All  of  these  are  necessary,  and  the  appropriate  level  and  balance  among  them  is  es- 
sential. 

In  response  to  the  new  NSF  program,  but  also  to  assure  that  these  advances  are 
sustained  and  extended,  the  Cornell  Theory  Center  is  increasing  its  collaborations 
with  computer  science  and  computational  mathematics  to  meet  the  challenge  of  this 
re-tooling.  We  are  therefore  creating  a  National  Alliance  for  Advanced  Computa- 
tional Infrastructure.  This  will  comprise  a  new  network  of  collaborating  organiza- 
tions that  will  bring  their  varied  talents  and  resources  to  the  Alliance  along  with 
the  world-class  supercomputing  infrastructure  of  the  Theory  Center  and  its  strong 
and  experienced  professional  staff.  Our  partners  are  of  world-class  stature  in  scal- 
able computing;  in  related  technologies,  such  as  virtual  reality  and  computer  graph- 
ics; in  all  areas  of  computer  science  that  relate  to  parallel  computing;  in  computa- 
tional mathematics;  in  forefront  applications. 

You  have  asked  how  the  new  competition  has  affected  our  Center.  It  has  certainly 
diminished  the  strong  sense  of  collaboration  with  the  others  that  went  by  the  rubric 
of  "MetaCenter."  It  has  created  a  sense  of  uncertainty  about  the  future  in  the  minds 
of  some  of  our  professional  staff.  Naturally  the  realization  that  this  is  a  very  serious 
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competition  with  very  high  stakes  has  generated  an  intense  concentration  on  win- 
ning, distracting  us  from  our  normal  mission. 

Nevertheless,  we  share  a  commitment  to  the  idea  that  a  Partnership  of  the  kind 
that  we  envision  is  a  logical  next  step  for  the  Cornell  Theory  Center,  one  that  will 
serve  the  best  interests  of  the  community  devoted  to  computational  science  and  the 
broader  community  of  research  in  basic  science. 

Mr.  BOEHLERT.  Thank  you,  very  much. 
Dr.  Smarr? 

STATEMENT  OF  DR.  LARRY  L.  SMARR,  DIRECTOR,  NATIONAL 
CENTER  FOR  SUPERCOMPUTING  APPLICATIONS,  AND  PRO- 
FESSOR OF  PHYSICS  AND  ASTRONOMY,  UNIVERSITY  OF  ILLI- 
NOIS AT  URBANA-CHAMPAIGN,  ILLINOIS 

Dr.  Smarr.  Thank  you  very  much. 

I  am  Larry  Smarr.  I  am  the  Director  of  the  National  Center  for 
Supercomputing  AppHcations  at  the  University  of  Ilhnois  at  Ur- 
bana-Champaign.  I  am  also  Professor  of  Physics  and  Astronomy. 

I  would  like  to  make  a  few  points  that  have  been  brought  out  by 
the  questions  today.  I  will  leave  my  written  testimony  to  the 
record. 

I  think  one  of  the  interesting  things  as  I  look  back  over  the  last 
ten  years,  since  I  was  very  involved  in  getting  this  program  started 
in  the  first  place — I  was  one  of  those  folks  who  was  mentioned  ear- 
lier that  had  to  go  to  foreign  shores  to  get  access  to  American-built 
supercomputers — is  that  in  those  days  really  it  was  just  the  na- 
tional laboratories  that  had  these  capabilities. 

There  was  a  lot  of  uncertainty  among  folks  like  you  in  similar 
Congressional  hearings  about  whether  universities  would  in  fact  be 
able  to  rise  to  the  occasion,  and  whether  the  NSF  itself  would  be 
able  to  handle  something  that  traditionally  had  been  dealt  with  my 
the  DOE  or  by  the  military. 

I  think  that  the  fact  that,  say  at  our  center  alone  which  is  en- 
tirely run  by  the  University  of  Illinois,  the  University  staff,  grad- 
uate students,  post-docs,  faculty,  and  the  fact  that  we  have  had 
over  9000  users  during  that  last  ten  years  fi-om  500  different  insti- 
tutions, the  fact  that  we  have  corporate  partners  from  all  sectors 
of  the  economy  speaks  to  the  fact  that  the  National  Science  Foun- 
dation was  in  fact  able  to  rise  to  the  occasion. 

The  entire  national  network,  the  NSF  Net  that  we  take  for 
granted  today  and  that  is  all  commercialized  even,  arose  completely 
from  the  backbone  that  was  used  to  tie  those  centers  together  at 
56  kilobits.  That  speed  is  less  than  what  you  get  into  your  home 
today. 

Remember  the — I  mean,  ten  years  ago  we  were  talking  about 
this  national  backbone  being,  today,  of  lower  speed  than  ISD  into 
the  home.  And  in  fact  our  first  supercomputer  had  the  speed  that 
is  about  equal  of  what  Nintendo  is  going  to  bring  out  next  month 
in  their  new  Ultra-64.  The  8-year-olds  are  going  to  have 
supercomputers  in  their  basement  playing  videogames. 

That  is  what  this  dramatic  technology  does.  There  is  no  other 
technology  like  it.  At  our  center  alone  we  have  been  able  to  sustain, 
at  essentially  the  same  amount  of  dollars  per  year  spent  on  hard- 
ware, a  75  percent  compounded  annual  growth  rate  in  computer  ca- 
pacity. 
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We  have  gone  through  three  complete  generations,  architectures, 
from  the  original  shared  Vector  machines  to  the  massively  parallel, 
to  the  new  microprocessor  based  systems  that  are  going  to  combine 
shared  memory  and  ease  of  programming  with  the  scalability  of 
massively  parallel  machines. 

One  thing  I  would  like  to  read  into  the  record  is  the  appendix 
from  the  Hayes  Report — and  I  think  all  four  directors  want  that  to 
happen — that  give  the  major  accomplishments  of  the  entire  NSF 
Centers  Program  both  in  technology  development,  outreach,  and  in 
the  scientific  research  enablement  of  remote  users  throughout  the 
university  community.  There  is  a  lot  of  work  that  went  into  prepa- 
ration of  that  document  that  I  think  adequately  will  answer  some 
of  the  questions  about  the  successes  of  the  program. 

I  would  like  to  move  next  to  this  notion  that  it  is  all  about 
supercomputing.  That  certainly  was  the  case  ten  years  ago.  It  is 
not  the  case  today. 

The  centers  have  evolved  and  matured  and  their  role  within  the 
context  of  the  national  program  has  changed  dramatically. 

For  instance,  the  NSF  center  dollars,  the  Cooperative  Agreement 
we  are  talking  about,  the  $65  million  available  each  year  from  the 
Division  of  Advanced  Scientific  Computing,  at  our  Center  is 
matched  dollar  for  dollar  by  other  sources  that  grew  up  as  a  result 
of  having  that  money  there.  That  came  from  the  State.  It  comes 
from  corporate  partners.  It  comes  from  vendors.  It  comes  from 
other  agencies. 

So  that  there  has  grown  up  around  the  core  program  a  whole  set 
of  partners.  So  the  partnering  idea  that  we  are  seeing  in  the  solici- 
tation is  nothing  new.  It  has  organically  grown  out  of  the  success 
and  maturation  of  the  original  center  concept. 

The  network  is  now  accelerating  the  need  for  that. 

Furthermore,  from  the  very  beginning  it  was  clear  that  visualiza- 
tion and  now  Virtual  Reality  techniques  are  needed  to  be  able  to 
understand  the  massive  amount  of  numbers  that  are  generated 
from  the  supercomputers. 

So  Virtual  Reality  development  itself  has  become  a  major  aspect 
of  ours  and  other  centers,  as  well  as  software  development. 

Many  of  you  are  I  am  sure  aware  that  one  software  product, 
NCSA  MOSIAC,  that  came  from  our  Center  has  completely  trans- 
formed the  world  in  a  matter  of  just  a  couple  of  years. 

Before  this,  the  Worldwide  Web  was  something  I  do  not  think 
any  of  us  had  ever  heard  about.  With  the  ease  of  use  of  this  one 
software  product,  millions  of  users  worldwide  in  probably  just  18 
months  started  using  this  to  lay  the  basis  for  one  of  what  Wall 
Street  considers  the  most  successful  technology  transfer  program  in 
probably  the  history  of  federal  funding,  which  is  the  development 
of  companies  like  NetScape's  SPYGLAS,  the  licensing  of  over  40 
companies  of  this  product  including  MicroSoft,  and  now  this  battle 
between  NetScape  and  MicroSoft. 

All  of  that  came  from  one  set  of  mainly  students  working  at  our 
Center. 

So  the  ability  to  keep  America  on  the  cutting  edge  of  this  new 
world  of  networks,  and  software,  and  the  Web,  the  Centers  have 
played  a  decisive  role  in  all  of  that. 
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The  same  goes — and  I  will  not  go  into  detail — for  the  K  through 
12,  the  outreach  to  communities,  the  work  with  industrial  partners, 
and  indeed  the  Federal  Government.  There  are  16  Federal  agencies 
that  are  going  together  to  learn  how  to  help  in  the  streamlining  of 
the  Federal  Government  system  using  these  techniques  at  NCSA. 

So  what  is  wrong  with  the  current  picture? 

That  is  what  I  am  hearing. 

I  think  the  point  is  that,  with  networking  you  are  having  a  phase 
change  in  the  way  this  country  is  put  together,  whether  you  are  in 
business,  or  whether  you  are  in  the  government,  or  whether  you 
are  in  health  care,  or  whether  you  are  in  education,  or  whether  you 
happen  to  be  in  high-performance  computing. 

That  is,  that  distance  is  rapidly  disappearing. 

So  the  notion  that  a  Center  being  in  one  geographical  spot  is  be- 
coming an  anachronism.  In  fact,  what  is  going  to  happen  is  in  the 
way  you  see  all  the  modem  management  books  for  industry  emerg- 
ing is  that  of  Virtual  Teams  that  come  together,  go  apart,  are  sup- 
ported by  the  Network  technologies,  and  the  fact  that  we  have  now 
these  Lego-like  units  of  microprocessors  that  can  go  from  the 
desktop  machine  to  mid-level  machines  to  the  very  highest  leading 
edge  machines,  that  are  distributed  across  and  almost  like  a  sea 
of  processors  across  this  network. 

That  is  the  world.  It  has  nothing  to  do  with  this  solicitation.  The 
world  has  changed,  and  largely  because  of  the  success  of  the  first 
ten  years  of  the  program. 

The  way  I  interpret  the  NSF's  solicitation  is  that  they  are  now 
saying.  Given  this  new  world,  given  that  we  have  a  fixed  amount 
of  dollars  each  year  in  the  Division  of  Advanced  Scientific  Comput- 
ing, how  can  we  create  Virtual  Centers  that  take  advantage  of  this 
new  world  and  all  of  the  ideas,  all  of  the  smart  people  that  are  not 
in  Champaign-Urbana? 

How  do  they  become  part  of  the  Center? 

Right  now  the  fundamental  problem  is.  If  I  wanted  to  fund  an 
effort  that  was  at  some  other  university  directly,  rather  than  spend 
it  in  Champaign-  Urbana,  I  really  cannot  do  that  within  the  cur- 
rent structure  of  the  program. 

The  new  program  would  envision  that  a  portion  of  the  coopera- 
tive agreement  that  would  come  to  me  as  PI  at  my  center  would 
in  fact  be  spent  out  in  the  field  at  these  other  partnering  institu- 
tions and  not  at  the  center. 

I  think  it  is  only  when  you  can  actually  get  resources,  dollar  re- 
sources, out  of  the  partnering  centers  that  the  kind  of  tight  cou- 
pling that  Ed  Hayes  referred  to  actually  comes  into  being,  because 
once  you  are  doing  that,  you  also  have  the  people  at  those  other 
centers  that  are  involved  in  the  management  of  your  own  central 
center. 

I  think  that  that  shared  management  of  shared  brains  around 
the  table  from  a  number  of  the  very  best  places  in  the  country  is 
a  very  different  vision  than  the  current  one  in  which  there  is  sim- 
ply one  cooperative  agreement  that  has  been  in  one  place,  and  then 
there  are  other  programs. 

So  I  think  that  is  one  of  the  key  advantages. 

The  other  is  that  there  is  this  tendency  to  say,  well,  we  are  going 
from  four  centers  to  two  centers.  I  do  not  see  that  at  all. 
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Technically,  yes,  to  get  within  the  same  budget  money  to  put  out 
in  the  Partnership  Centers  you  may  well  have  to  reduce  the  num- 
ber of  leading  edge  centers,  but  you  are  gaining  other  centers  in 
the  process — namely,  the  partnering  centers. 

They  are  becoming  an  integral  part  in  a  virtual  teaming  ap- 
proach of  this  new  PACI  leading  edge  center.  So  we  could  end  up 
with  Centers  of  Excellence  in  Virtual  Reality,  and  Mid-Level  Per- 
formance, and  Reaching  Out  in  K  Through  12,  that  are  actually 
more  than  we  currently  have. 

The  expense  to  do  that  within  a  fixed  budget  is  to  reduce  the 
number  of  leading  edge  centers. 

All  I  can  say  is  that  the  solicitation  has  generated  more  new 
ideas,  more  good  feeling  among  the  community  at  large,  than  any- 
thing I  have  seen  in  ten  years.  I  think  that  the  kinds  of  concerns 
you  have  that  we  do  not  break  the  well-functioning  current  system 
are  well  placed,  and  v/e  have  to  be  very  careful  that  that  is  taken 
care  of  during  the  solicitation. 

The  best  protection  for  that  is  a  strong  merit  review  process.  I 
think  and  hope  that  the  committee  today  will  reinforce  the  appro- 
priateness that  merit  review  through  the  NSF  system  is  the  best 
way  to  make  sure  that  there  is  a  smooth  transition  and  that  none 
of  the  services  to  the  national  community  are  less,  but  in  fact 
greatly  increased. 

[The  prepared  statement  of  Dr.  Smarr  follows:] 
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The  Honorable  Steven  H.  Schiff 
Chairman,  Basic  Research  Subcommittee 
U.S.  House  of  Representatives 
Committee  on  Science 
Suite  2320  Raybum  House  Office  Building 
Washington,  DC  20515-6301 


Mr.  Chairman: 

Mr.  Chairman,  it  is  an  honor  to  be  here  this  afternoon.  I  am  going  to  talk  with  you 
today  about  the  National  Center  for  Supercomputing  Applications  at  the  University  of 
Illinois  at  Urbana-Champaign. 

Specifically,  I  would  like  to  talk  with  you  about  how  NCSA  is  participating  in  the 
National  Science  Foundation's  Partnerships  for  Advanced  Computational  Infrastructure 
solicitation. 

In  Section  1  of  my  written  testimony  I  have  provided  you  with  an  overview  of  NCSA's 
programs.  This  section  contains  information  on  NCSA's  history,  funding,  research 
tools,  technology  and  knowledge  transfer,  our  Fortune  500  industrial  partners,  and  the 
Federal  Consortium. 

Section  2  provides  a  response  to  two  of  the  questions  you  mentioned  in  your  letter  of 
invitation. 

Attachment  1  highlights  major  scientific  accomplishments  of  the  NSF  Supercomputer 
Centers  Program. 

Again,  thank  you  Chairman  Schiff  for  the  opportunity  to  participate  in  today's  hearing. 


Eh-.  LafTyL-  Smarr 
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Section  1 — Overview  of  the  National  Center  for  Supercomputing  Applications 

Background  

Under  the  leadership  and  guidance  of  the  National  Science  Foundation  the  National  Center  for  Supercomputing 
Applications  (NCSA)  at  the  University  of  Illinois  at  Urbana-Champaign  has  evolved  into  a  scientific  research  center 
built  around  a  national  services  facility.  All  aspects  of  NCSA  are  carried  out  within  a  major  research  university.  At  the 
founding  of  the  NSF  Centers  program  10  years  ago,  there  were  those  who  questioned  whether  a  university  could 
successfully  manage  such  a  critical,  national-scale  program.  History  has  shown  that  the  integration  of  the  program  into 
the  university  environment  has  resulted  in  much  of  the  uniqueness  and  synergy  leading  to  rapid  advances  in  scientific 
knowledge,  application  software  development  and  technology  transfer. 

In  its  leadership  role,  NCSA  is  developing  and  implementing  a  national  strategy  to  create,  use,  and  transfer  advanced 
computing  and  communication  tools  and  information  technologies.  These  advances  serve  the  center's  diverse  set  of 
constituencies  in  the  areas  of  science,  engineering,  education,  and  industry. 

When  the  center  opened  to  the  research  community  in  1986,  researchers  using  the  supercomputers  were  from  the 
traditional  disciplines  of  physics,  chemistry,  materials,  and  astrophysics.  But  researchers  in  other  disciplines  quickly 
realized  they  could  also  transform  their  fields  by  using  supercomputing.  Today's  growth  areas  include  biology,  medicine, 
environment,  computer  science,  commercial  and  fmancial  databases,  and  networked  World  Wide  Web  multimedia  asset 
management 

Academic  and  industrial  users  from  across  the  nation  have  utilized  NCSA's  leading  edge  supercomputers  to  enable 
advances  in  their  research.  As  a  result  of  rapid  changes  in  computing  technology,  NCSA  has  been  able  to  continually 
increase  supercomputer  capacity  and  capability  while  keeping  dollar  expenditures  relatively  constant.  Specifically, 
NCSA  capacity  of  supercomputing  time  made  available  to  national  users  has  increased  at  75  percent  per  year,  resulting 
in  a  1,000  fold  increase  in  capacity  between  the  center's  opening  in  1986  and  the  end  of  the  current  cooperative 
agreement  in  FY97.  To  utilize  this  rapid  growth  in  computing  power,  NCSA  has  worked  with  users  to  develop  and 
migrate  application  codes  through  three  distinct  phases  of  supercomputer  architectures:  shared  memory  vector 
processors;  massively  parallel  processors;  and  scalable  shared  memory  RISC  microprocessors.  During  this  period,  nearly 
9,000  users  from  more  than  500  academic  institutions  and  corporate  laboratories  in  all  50  states  have  made  use  of 
NCSA's  computational  resources.  In  addition,  hundreds  of  university  level  courses  in  computational  science  and 
engineering  have  used  NCSA  supercomputers.  See  Attachment  1  (Major  Accomphshments  of  the  NSF  Supercomputer 
Centers  Program),  detailing  scientific  highhghis  and  user  accomplishments.  The  attachment  is  being  jointly  submitted  by 
all  four  NSF  centers. 

NCSA  is  dedicated  to  improving  the  competitive  position  of  American  industry  by  partnering  with  leading  corporations 
in  a  variety  of  sectors  in  the  US  economy  to  partner  with  NCSA.  These  Fortune  500  companies  have  selected  NCSA  lo 
learn  about  all  aspects  of  cutting-edge  hardware  and  software,  virtual  prototyping,  visualization,  networking,  and 
databases. 

NCSA  works  closely  with  computer  vendors  and  national  leaders  of  academic  computer  science  to  bring  users  the  most 
advanced  methods  in  high-performance  scalable  computing.  The  center  currently  maintains  these  supercomputers: 
Hewlett-Packard/CONVEX  Exemplar,  Silicon  Graphics  POWERCHALLENGEarray,  and  Thinking  Machines  CM-5. 
NCSA  has  allowed  its  vendor  partners  to  gain  access  to  national  leaders  in  science  and  engineering  and,  as  a  result,  the 
vendors  have  produced  machines  that  are  more  useful  to  the  American  research  community. 

Funding  Sources 

NCSA  relies  on  its  cooperative  agreement  from  NSF  as  the  foundation  of  the  center's  efforts.  However,  NCSA  has 
aggressively  leveraged  the  core  NSF  funding  providing  a  dollar  to  dollar  match  from  other  sources.  This  has  resulted  in 
an  additional  $132.8  million  over  10  years  being  added  to  NCSA's  program  beyond  the  NSF  core  funding.  Sources  of 
these  leveraged  funds  are  other  federal  agencies,  corporate  partners,  the  University  of  Illinois,  and  the  State  of  Illinois. 
This  additional  funding  has  been  critical  in  expanding  NCSA's  offerings  beyond  a  central  supercomputing  facility  and 
for  developing  world  recognized  staff  in  software  development,  scientific  applications,  and  virtual  environments. 


Dr.  Lany  L.  Smart,  NCSA 
Congressional  Teslimony 
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New  Tools  for  Computational  Science  and  Engineering 


To  support  the  large  number  of  remote  users  NCSA  has  developed  a  broadly  based  scientific  research  center  of 
application  researchers  housed  in  the  Beckman  Institute  at  the  University  of  Illinois  at  Urbana-Champaign.  These  staff 
members  act  as  a  bridge  to  their  disciplines  and  have  been  a  constant  source  of  innovation  in  the  use  of  new  technology 
10  empower  new  applications.  NCSA  has  application  scientists  in  the  fields  of  astrophysics,  biomedical  science, 
chemistry,  chemical  engineering,  computational  biology,  computational  fluid  dynamics,  condensed  matter  physics,  earth 
sciences,  gravitational  sciences,  humanities/fine  arts,  information  systems,  mathematics,  computer  science,  radio 
astronomical  imaging,  and  structural  engineering.  For  example,  one  of  NCSA's  application  scientists  is  PI  on  the  joint 
agency  (NSF,  ARPA,  NASA)  Digital  Library  Initiative  at  the  University  of  Illinois  at  Urbana-Champaign. 

NCSA's  Virtual  Environments  Laboratory  is  one  of  the  world's  most  advanced  virtual  reality  (VR)  research  laboratories 
available  to  academic  and  industrial  researchers.  The  lab  consists  primarily  of  three  projection-based  modes  of  VR — the 
CAVE™  and  the  ImmersaDesk™  (both  developed  by  the  Electronic  Visuahzation  Laboratory  [EVL]  at  the  University  of 
Illinois  at  Chicago)  and  the  NlIAVall  (developed  by  EVL  in  collaboration  with  NCSA  and  the  Laboratory  for 
Computational  Science  and  Engineering  at  the  University  of  Minnesota). 


Technology  Transfer 


In  addition  to  developing  new  software  tools  and  making  them  freely  available  to  the  academic  community,  NCSA  has 
had  a  significant  impact  on  commercial  software  development  and  products.  NCSA  is  proud  of  its  proven  record  of 
catalyzing  new  products  and  companies.  NCSA  software  developers  who  joined  InterCon  Systems  Corp.  in  1987  started 
a  long-term  product  line  based  on  NCSA  Telnet™ .  Similarly,  former  NCSA  developers  founded  Spyglass  Inc.  in  1990 
with  software  based  on  NCSA  Image™.  The  1992  release  of  the  World  Wide  Web  network  browser  NCSA  Mosaic 
created  millions  of  Web  users  and  both  laid  the  technology  groundwork  and  created  a  market  for  subsequent 
commercialization.  This  commercialization  was  accomplished  via  two  routes.  First,  NCSA  software  developers  joined 
companies  such  as  Microsoft,  Spry,  Spyglass,  and  Netscape.  Second,  UlUC/NCSA  intellectual  property  is  licensed 
through  Spyglass  to  40  companies,  and  incorporated  into  more  than  100  products.  The  economic  results  are  represented 
by  the  example  of  just  two  of  these  companies.  Spyglass  and  Netscape,  which  in  the  past  twelve  months  have  attained  a 
market  valuation  as  high  as  $5  billion. 


Knowledge  Transfer 


NCSA  understands  that  transferring  necessary  technical  skills  and  information  is  vital  for  prolonged  growth  and  success 
of  any  research  activity.  This  transfer  is  accomplished  by  providing  computational  science,  networking  technologies,  and 
information  technology  training  to  fellow  researchers,  students,  other  educators,  lifelong  learners,  government  and 
community  organizations,  and  members  of  the  business  community. 

NCSA  addresses  science  and  curriculum  issues  through  programs  designed  not  only  to  give  a  technical  jump-start  to  a 
school  but  also  to  provide  continuing  support  to  educators  and  their  administrators.  Current  major  programs  include:  the 
Education  Affiliates  (EA)  program,  the  Resource  for  Science  Education  (RSE)  program,  the  Networking  Infrastructure 
for  Education  (NIE)  project,  EduLinK-12  (IlUnois  Education  Link),  Tech  Corps  Illinois,  and  Illinois  Learning  Mosaic 
(ILM,  a  World  Wide  Web-based  educational  information  resource). 

NCSA's  community  networking  and  technology  transfer  program  works  with  various  communities  to  deploy  computing 
and  communications  technologies  and  applications  at  the  local,  state,  regional,  and  national  levels.  The  center's  CCNet 
(Champaign  County  Network)  collaboration  with  its  host  community  has  received  national  recognition  as  an  innovative 
application  of  the  National  Information  Infrastructure  and  as  a  model  for  other  communities.  Chicago  Mosaic,  a 
collaborative  project  with  the  City  of  Chicago,  is  investigating  the  use  of  World  Wide  Web  technologies  lo  communicate 
among  city  government,  the  citizens  of  that  community,  and  the  rest  of  the  world. 


Dr.  Lany  L.  Sman,  NCSA 
Congressional  Testimony 


116 


Fortune  500  Industrial  Partners 


Current  NCSA  industrial  partners  include  Allstate,  AT&T,  AMR,  Caterpillar,  Dow  Chemical,  Eastman  Kodak  Co.,  Eli 
Lilly  &  Co.,  FMC,  J. P.  Morgan,  Motorola  Inc.,  Phillips  Petroleum  Co.,  Schlumberger,  Sears,  Roebuck  and  Co.,  Tribune 
Co.,  and  United  Technologies.  Since  1986  sixteen  partners  have  added  540,672,913  to  NCSA's  programs  in  order  to 
learn  about  the  evolving  high  performance  computing  technologies,  to  apply  them  to  solving  breakthrough  industrial 
applications,  and  to  support  technology  research  at  NCSA.  Their  investment  in  this  program  has  enabled  each  company 
to  enhance  technical  and  management  skills  and  to  explore  the  leading  edge  technologies  which  are  needed  for  them  to 
help  stimulate  this  country's  economic  viability.  As  a  result  of  their  successes  in  this  program,  NCSA  and  the  University 
of  Illinois  at  Urbana-Champaign  have  gained  a  reputation  for  excellence  among  our  nation's  industrial  leadership. 


NSF/NCSA  WWW  Federal  Consortium 


A  group  of  16  US  Federal  agencies,  assembled  through  the  efforts  of  the  National  Science  Foundation,  came  to  NCSA  to 
learn  how  to  apply  new  technologies  in  their  efforts  to  stfeamline  internal,  cross  agency,  and  multi-level  government 
processes  and  communication.  Member  agencies  include  the  following:  Nuclear  Regulatory  Commission;  National 
Library  of  Medicine;  Cenual  Intelligence  Agency;  Intelligence  Services  (COSPO);  National  Science  Foundation; 
National  Institute  of  Standards  and  Technology;  Department  of  Education;  Defense  Technical  Information  Center; 
National  Oceanic  and  Atmospheric  AdminisU'ation;  National  Security  Agency;  National  Cancer  Institute;  National 
Institutes  of  Health;  National  Aeronautics  and  Space  Administration;  US  Geological  Survey;  National  Biological  Service 
and  Bureau  of  the  Census. 
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Section  2— Impact  of  the  NSF  Solicitation  on  NCSA 


Q:  "What  impact  is  the  new  solicilation  having  on  the  current  centers?" 

A:  NCSA  has  partnered  with  a  number  of  groups  during  the  past  10  years  including  US  industry  and  other  R&D 
facilities.  In  response  to  the  solicitation  NCSA  is  expanding  the  scope  of  its  partnerships  and  subsequent  projects. 
Partnering  with  other  institutions  is  a  key  component  of  the  NSF  solicitation.  Therefore,  NCSA  is  aggressively  pursuing 
exciting  new  working  relationships  with  computational  scientists,  computer  scientists,  academic  institutions,  R&D 
laboratories  and  regional  centers  to  prototype  the  US  computational  and  information  infrastructure  of  the  21st  Century. 

Also,  NCSA  is  developing  new  and  innovative  ways  to  manage  a  larger  number  of  partners  and  highly  complex  projects. 
The  end  result  will  be  a  stronger,  national-scale  laboratory  for  the  creation  and  management  of  "virtual  teams,"  which  are 
becoming  essential  to  the  emerging  knowledge  worker  communities  in  American  industry,  government,  universities, 
health  care,  and  education. 

The  solicitation  has  not  led  to  a  negative  impact  on  the  current  daily  operations  at  NCSA.  The  PACI  mission  has 
heightened  vendor  interest  in  increasing  their  commitment  and  partnerships  with  NCSA.  In  addition,  the  solicitation  has 
led  to  a  closer  working  relationship  with  NCSA's  end  users. 

In  conclusion,  the  solicitation  has  challenged  NCSA  to  raise  its  sights  to  see  what  the  center  can  further  accomplish  to 
accelerate  national  objectives. 


Q:  "How  are  the  current  centers  working  to  participate  in  the  new  solicitation  proposals?" 

A:  As  requested  by  NSF,  NCSA's  management  is  writing  a  preproposal  based  on  the  PACI  mission  statement  and  NSF 
guidelines.  NCSA  is  competing  to  become  one  of  NSF's  leading  edge  sites  in  the  PACI  program. 

NCSA  has  taken  the  following  steps  to  seek  out  partnerships: 

•  Discussed  NCSA's  new  vision  with  potential  parmers  (including  DOE  laboratories,  universities,  consortia,  research 
groups,  other  NSF  research  centers,  and  state  and  regional  centers) 

•  Solicited  innovative  ideas  from  potential  partners 

•  Matched  NCSA's  vision  with  potential  partners  and  their  ideas 

•  Aligned  all  goals  and  objectives  with  PACI  mission  statement 

NCSA  has  found  the  challenge  of  the  PACI  mission  exciting.  The  center  has  also  witnessed  many  innovative  ideas 
emerge  from  the  broader  community  stimulated  by  the  solicitation.  As  long  as  this  process  remains  under  the  firm 
control  of  NSF's  merit  review  system,  NCSA  believes  the  result  will  be  a  stronger,  more  broadly  based  national 
program. 

In  conclusion,  NCSA  believes  that  the  PACI  program  is  critical  to  the  future  of  America's  research  system.  The  PACI 
program  will  allow  for  critical  early  experience  with  the  advanced  computational  infrastrucnire  that  will  support  21st 
Century  scientific  and  engineering  research. 
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High  Performance  Computing  Infrastructure  and  Accomplishments 


Introduction 

The  NSF  Centers  Program  was  established  to  provide  access  to  high  performance 
computers  (supercomputers)  for  the  broad  Science  and  Engineering  Research  Com- 
munity. The  program  has  evolved  from  one  comprising  independent,  competitive, 
and  similar  computer  centers  to  one  including  more  cooperative  and  diverse  activi- 
ties. Coordinating  the  mission  of  the  individual  centers  has  increased  the  diversity 
of  computer  architectures  available  to  the  research  community,  and  has  accelerated 
outreach  to  segments  of  the  community  which  had  not  before  been  able  to  use  the 
power  of  high  performance  computers.  At  the  same  time,  competition  between  cen- 
ters has  been  managed  by  NSF  and  its  advisory  committees  to  the  advantage  of  the 
engineering  and  science  communities  which  the  program  was  established  to  serve. 

Building  on  each  center's  tradition  of  providing  a  stable  source  of  computer  cycles 
for  a  large  community  of  scientists  and  engineers,  the  centers  have  evolved  into  a 
unique  resource  to  test  which  diverse  computer  architectures  best  match  the  most 
demanding  problems  posed  by  the  community  of  university  researchers  and  to  de- 
velop the  necessary  supporting  software  and  algorithms.  For  example,  this  approach 
has  enabled  the  centers  to  test  which  applications  can  be  efficiently  served  on  newly 
developed  systems  using  clusters  of  the  new  generation  of  workstations  that  are 
now  being  introduced.  Such  experiments  are  enabled  by  the  open  environment 
characteristic  of  the  program. 

During  the  first  decade  of  the  centers  program,  major  improvements  in  the  delivery 
of  high  performance  computing  have  been  developed,  mairUy  by  American  com- 
puter researchers  and  companies.  But  advances  in  computing  technology  have  been 
matched  in  equal  measure  by  improvements  in  computer  networking,  and  as  a  con- 
sequence the  NSF  Centers  have  been  a  primary  focus  for  accelerating  the  evolution 
of  the  Internet  via  NSFnet,  NREN,  and  the  still  evolving  broad-band  width  tech- 
nology. 

In  this  appendix,  we  provide  detailed  examples  of  the  centers  activities. 
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High  Performance  Computing  Infrastructure  and  Accomplishments 

Important  Technology  Accomplishments 
High  performance  computing 


Originally  set  up  in  1985  to  provide  national  access  to  traditional  supercomputers, 
the  NSF  Centers  have  evolved  to  a  much  larger  mission.  The  Centers  now  offer  a 
wide  variety  of  high  performance  architectures  from  a  large  array  of  American  ven- 
dors. No  longer  just  adopting  technology  from  the  national  labs,  the  NSF  Centers 
Program  has  become  a  pioneering  vanguard  of  technology  -  a  model  for  other  agen- 
cies with  a  vested  interest  in  the  high  performance  computing  to  emulate.  This  to- 
day is  dominated  by  research  efforts  in  software,  with  vital  collaborations  with  com- 
puter scientists,  focusing  on  operating  systems,  compilers,  network  control,  mathe- 
matical libraries,  and  programming  languages  and  environments.  The  feedback  to 
the  leading  US  vendors  is  increasing  the  usefulness  of  their  product  offerings  to  the 
scientific  and  engineering  communities,  while  making  them  more  competitive. 

Supercomputer  Usage  at  NSF  Centers 


Fiscal  Year 

Active  Users 

Usage  in  CPU 
Hours 

1986 

1336 

29,485 

1987 

3,299 

95,751 

1988 

5,042 

121,615 

1989 

5,967 

165,960 

1990 

7,357 

250,628 

1991 

7,723 

361,073 

1992 

8,252 

398,931 

1993 

7,735 

910,088 

(Usage  is  in  normalized  CPU  hours,  based  on  comparative  performcmce  tests.  The  astounding  leap  in  ca- 
pacity in  1993  is  mainly  a  result  of  the  introduction  of  new  computing  architectures  to  solve  the  most 
demanding  of  computational  problems  — the  Grand  Challenges.  The  slight  decrease  in  the  number  of 
users  is  the  result  of  a  concerted  effort  by  the  Centers  to  assist  many  of  their  users  with  small  memory  or 
CPU-time  requirements  to  meet  their  computational  needs  by  the  increasingly  powerful  workstations  of 
the  mid-90's.  The  greatest  benefactors  of  the  increase  in  massively  parallel  cycles  are  the  scientists  and 
engineers  addressing  the  problems  with  the  greatest  computing  demands) 

Architectures  and  Vendors 
The  national  community  has  been  offered  access  to  a  wide  and  frequently  updated  set 
of  high  performance  architectures  since  the  beginning  of  the  NSF  Supercomputer 
Centers  Program.  The  current  rate  of  change  of  the  types  of  architectures,  and  the 
number  of  vendors  offering  them,  is  probably  near  an  all  time  high.  We  are  in  a  pe- 
riod of  ferment  which  the  science  and  engineering  communities  sort  out  the  choices 
for  finding  an  architecture  that  matches  their  various  computational  problems.  A  list 
of  architectures  that  the  NSF  Centers  Program  has  offered  would  include:  single  and 
clustered  high  performance  workstations  or  workstation  multiprocessors,  minicom- 
puters, graphics  supercomputers,  mainframes  with  or  without  attached  processors  or 
vector  units,  vector  supercomputers,  and  SEvID  and  MIMD  massively  parallel  proces- 
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sors.  Similarly,  the  list  of  current  vendors  whose  top  machines  have  been  made 
available  would  include  IBM,  DEC,  Hewlett-Packard,  Silicon  Graphics,  Sun  Microsys- 
tems, Cray  Research,  Convex  Computer,  Intel  Supercomputer,  Kendall  Square, 
Thinking  Machines,  nCUBE,  Alliant,  Floating  Point  Systems,  ETA,  Stellar,  Ardent, 
and  Stardent. 

Center  Program  Chronology 


FY 

Milestone  or  Event 

Description 

1986 

5  NSF  Supercomputer  Centers  become 
operational 

Cornell  1  heory  Center 

National  Center  for  Supercomputing  Applications 

Pittsburgh  Supercomputing  Center 

San  Diego  Supercomputer  Center 

John  von  Neumann  Center 

1988-1989 

Renewal  Review 

1989 

Renew  4  NSF  Supercomputer  Centers 
(5yrs.) 

CTC,  NCSA,  l^C,  SDSC 

1990-1992 

ASC  Advisory  Committee  report 
completed 

strong  recommendations  for  adding  parallel  systems  to 
accompany  the  stable,  production  vector  systems 

1991 

8  Vector  Supercomputers  operating 
3  Scalable  parallel  systems  operating 

IBM  3090                 6  processors  (2) 
Cray  YMP               8  processors  (2) 
Alliant  FX80           8  processors 
Cray  2S                    4  processors 
Cray  YMP               4  processors 
Convex  C240           4  processors 
Intel  iPSC/860              32  processors^ 
TMC,  CM2                    32,000  processors  (2)''' 

1992 

7  Vector  Supercomputers  operating 
9  Scalable  parallel  systems  operating 

Joint  Planning  Initiated 

Cray  systems  remain 

IBM  ES9000/900  (Upgrade  3090) 

Alliant  2800  (upgracie  FX80) 

Convex  C3880         8  processors  (Upgrade  C240) 
Intel  iPSC/860  upgrade      64  processors)t 
nCUBE2                        128  processors! 
TMC,  CMS                    512  processorsT 
KSRl                             64  processors! 
DEC  Workstation  Ouster  (2)t 
IBM  Workstation  Cluster 
IBM  PVS                      32  processors 

Initial  meeting  at  SDSC  Fall  92 

1993 

Joint  Activities  Began 

6Vector  Supercomputers  operating 
13  Scalable  parallel  systems  operating 

Meeting  at  PSC 
First  joint  proposals  to  PPRP 
First  of  Joint  projects 
Cray  C90                 16  processors 
(other  Crays,  IBM,  Convex  stay) 
Intel  Paragon                400  processors  upgrade  t 
KSRl  upgrade               160  processors! 
Hewlett  Packard  Cluster^ 
MasPar  2                      16,000  processors! 
IBM  SPl                        64  processors! 
Cray  T3D                      512  processors! 

1994 

Joint  Activities 

4  Vector  Supercomputers  operating 
14  Scalable  parallel  systems  operating 

Meeting  at  CTC 
Metacenter  Regional  Alliances  -  Mar  94 
Expansion  of  joint  projects 
One  YMP  changed  to  C-90,  others  the  same 
IBM  SP2  upgrade         512  processors  upgrade  SPl! 
Cray  T3D                      512  processors  ! 
Convex  Exemplar         8  Nodes! 
SGI  Challenge               32  Nodes 

^  Majority  of  funding  provided  by  other  Federal  agency  (ARPA,  NIH)  or  state, 
t  Donated  in  full  or  in  part  by  the  manufacturer  for  extended  evaluation 
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The  Centers  Program  provides  a  stable  supply  of  vector  computing  cycles  needed  by 
the  research  community  while  investing  in  significant  capacity  of  scalable  parallel 
systems,  capable  of  ultimately  growing  to  a  size  necessary  for  full  scale  grand  chal- 
lenge problems.  While  the  numbers  of  Vector  Supercomputers  has  decreased,  the 
computing  power  represented  by  that  group  in  fact  increased  substantially.  How- 
ever, the  increase  in  the  scalable  parallel  systems  was  much  larger,  reflecting  the 
growth  potential  of  this  type  of  computing  platform  and  a  strong  NSF/ARPA  part- 
nership. 

National  Access  to  Vector  Multiprocessors 

The  NSF  Supercomputer  Centers  established  in  the  mid-1980s  brought  access  to  state- 
of-the-art  supercomputers  for  the  first  time  in  at  least  15  years.  Indeed,  in  the  1960s, 
only  a  few  universities  had  such  access.  This  opening  of  universal  access  led  to  an 
unprecedented  increase  in  the  number  of  researchers  and  universities  involved  in 
advancing  the  frontiers  of  scientific  and  engineering  research  by  using  high  perfor- 
mance computing.  By  the  early  1990s,  some  15,000  researchers  in  over  200  universi- 
ties had  used  one  of  the  Cray  Research  vector  multiprocessors  or  the  IBM  vector 
mainframe  in  one  of  the  NSF  centers.  This  wide  pool  of  computational  researchers 
made  it  possible  for  the  center's  program  to  begin  to  respond  to  the  demand  for  paral- 
lelism that  had  been  developed  in  the  Computer  Science  Community,  and  adopted 
by  the  most  adventurous  user.  The  90's  saw  the  NSF  center's  program  substantially 
widen  its  range  of  architectural  offerings. 

Achieving  Production  Parallelism 

The  Cornell  Theory  Center  (CTC)  became  the  first  member  Center  of  the  NSF  Meta- 
Center  to  achieve  production  parallelism  on  a  vector  supercomputer,  with  over  1/3 
of  its  vector  supercomputer  cycles  used  for  parallelism  in  1989.  CTC  integrated  its  two 
(ES/3090  600)  vector  supercomputers  using  a  special  200  mbyte/sec  hardware  inter- 
face, allowing  parallel  jobs  the  potential  of  executing  across  12  vector  processors. 
Users  used  a  shared-memory  parallel  FORTRAN  developed  by  IBM  in  a  joint  project 
with  the  Theory  Center. 

Early  Migration  to  the  UNIX  Operating  System 

During  the  early  and  mid-1980's  the  UNIX  operating  system  was  widely  viewed  as 
inappropriate  for  supercomputers  for  reasons  of  performance,  system  management 
tools,  application  development  and  measurement  mechanisms,  and  security.  Cray 
supercomputers  were  run  mostly  with  operating  systems  that  were  designed  at  na- 
tional laboratories  (LLNL  and  LANL  in  particular)  and  this  required  extensive  local 
software  support.  In  1987,  NCSA  became  the  first  major  supercomputer  center  to  mi- 
grate its  Cray  supercomputer  from  CTSS  (Cray's  proprietary  time-sharing  system)  to 
UNICOS,  a  UNIX-based  operating  system  developed  at  Cray  Research  for  its  super- 
computers. This  move  to  UNIX  was  the  beginning  of  a  merger  between  computa- 
tional science  and  computer  science,  because  most  computer  science  research  in- 
volved the  UNIX  operating  system  at  that  time.  Coincidentally,  CTC  was  the  first  site 
to  run  IBM's  high  performance  UNIX  system  on  its  ES/3090  and  ES/9000  main- 
frames in  production. 
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Early  Access  to  Massively  Parallel  Computers 

Beginning  in  1985,  CTC  provided  experimental  scalable  parallel  machines,  first  the 
FPS  T-series  and  an  iPSC/1  parallel  system,  to  its  user  community.  In  1988,  with  the 
installation  of  its  iPSC/2  with  32  processors,  CTC  made  early  scalable  computing 
available  for  production  use  by  the  national  community.  Massively  parallel  comput- 
ing was  introduced  to  the  research  community  beginning  with  NCSA's  CM-2  in 
1989.  Each  Center  provided  early  access  to  new  generation  MPPs.  The  CTC  was  the 
first  site  to  install  an  IBM  SPl  and  SP2,  PSC  installed  the  first  Cray  T3D,  and  early 
CMS  (NCSA),  Paragon(SDSC),  Ncube(SDSC),  and  Kendall  Square  (CTC)  machines 
were  installed.  Early  access  to  these  machines,  enabled  by  support  from  ARPA  and 
NIH,  allowed  pioneering  users  to  explore  the  benefits  of  fine-grained  parallelism. 
Each  Center  worked  with  the  user  community  and  the  vendors  to  develop 
application  codes,  which  could  then  be  ported  to  other  platforms.  In  1992,  the  CM-5 
was  added  to  the  program  at  NCSA  as  the  largest  distributed  memory  parallel  super- 
computer available  to  the  national  academic  and  industrial  communities.  From 
1992  to  the  present  NCSA  has  worked  closely  with  national  users  and  the  computer 
science  community  to  create  a  wide  range  of  512-way  parallel  application  codes  that 
can  in  1995  be  moved  to  other  large  MPP  architectures  such  as  the  T3D  at  PSC,  the 
Intel  Paragon  at  SDSC,  or  the  IBM  SP-2  at  CTC. 

Superlinear  speedup  on  heterogeneous  processors 

In  1991,  PSC  was  the  first  site  to  distribute  code  between  a  massively  parallel  machine 
(TMC-CM2)  and  a  vector  supercomputer  (Cray  YMP),  linked  by  a  high  speed  channel 
(HiPPI).  Experiments  on  applications  as  diverse  as  molecular  dynamics,  medical 
imaging,  chemical  flowsheeting  and  gene  sequence  alignment  showed  superlinear 
speedup  (the  applications  on  the  linked  system  ran  more  than  twice  as  fast  on  each 
system  separately).  This  formed  part  of  the  motivation  for  heterogeneous  comput- 
ing, as  later  embodied  in  the  tightly  coupled  Cray  T3D/C90  systems.  PSC's  T3D  was 
the  first  shipped  anywhere.  PSC  developed  a  set  of  codes  for  transferring  data  be- 
tween the  Cray  and  CM2  which  later  enabled  them  to  coitununicate  between  the  T3D 
and  C90  at  speeds  superior  to  what  was  available  from  the  vendor. 

A  similar  superlinear  speedup  was  obtained  on  the  CASA  gigabit  testbed  set-up  in- 
cluded parts  of  two  supercomputers  (64  nodes  of  the  528-node  Intel  Delta  system  at 
Caltech  and  one  processor  of  the  CRAY  C-90  at  SDSC,  150  miles  away);  two  HiPPI- 
SONET  gateways;  and  a  SONET  wide-area  link  between  San  Diego  and  Pasadena, 
which  is  itself  a  prototype  undergoing  tests  in  a  collaboration  between  MCI  and  Pa- 
cific Bell.  Using  an  environment  for  distributed  parallel  computing  called  Express  (a 
product  of  ParaSoft  Corporation),  Aron  Kuppermarm  and  Mark  Wu  (Dept.  of 
Chemistry,  CalTech)  did  a  test  calculation  of  the  reaction  of  atomic  hydrogen  with 
molecular  heavy  hydrogen  (deuterium)  at  a  total  energy  of  2.5  eV.  This  problem  had 
taken  100  hours  to  solve  on  the  SDSC  CRAY  Y-MP  a  year  earlier.  On  the  new  C-90,  it 
took  17  hours.  On  the  Delta  alone,  it  took  16.  But  when  the  problem  was  distributed 
between  the  C-90  and  the  Delta,  the  whole  problem  was  solved  by  the  two  machines 
in  just  under  five  hours,  a  factor  of  3.3  faster  than  it  could  be  done  on  either  ma- 
chine alone. 
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Workstation  Clusters 

Given  historical  trends  showing  much  more  rapid  improvement  in  microprocessor 
technology  than  in  vector  technology,  many  Centers  began  exploring  workstation 
clusters.  In  1990,  NCSA  examined  the  usage  of  the  Cray  Y-MP  and  determined  that  a 
significant  amount  of  capacity  could  be  gained  by  moving  appropriate  applications  to 
scalar  RISC  processors.  Based  on  this  study  and  on  predictions  that  microprocessor 
technology  would  surpass  vector  technology  by  the  mid-1990's,  NCSA  set  up  an  IBM 
RS/6000  cluster  as  a  farm  of  processors  used  as  stand-alone  compute  servers.  Begin- 
ning in  1991,  CTC  did  pioneering  work  with  IBM  on  clustered  RS/6000  workstations 
with  high-speed,  proprietary  communications  links,  including  an  experimental  opti- 
cal switch.  The  information  gained  during  this  joint  project  was  used  to  guide  the  de- 
sign of  the  SP  systems.  Also  in  1991,  PSC  set  up  a  cluster  of  DEC  workstations,  and 
working  with  Florida  State  University,  significantly  enhanced  queuing,  accounting 
and  control  software  and  also  integrated  the  AFS  file  system  into  this  environment. 
These  clusters  all  served  as  computing  resources  and  also  as  platforms  for  the  devel- 
opment of  distributed  memory  message  passing  codes.  These  projects  generated  high 
interest  in  industry,  and  NCSA  trained  several  dozen  industrial  sites  on  the  integra- 
tion, operation,  and  management  of  clusters  of  microprocessors.  NCSA  is  now  mov- 
ing to  establish  dusters  of  shared  memory  workstation  multiprocessors  from  Con- 
vex/HP and  SGI,  while  PSC  will  develop  applications  for  Cray's  new  offering,  the  J90. 
The  CTC  was  given  supplemental  funding  by  NSF  to  build  its  cluster  to  32  proces- 
sors, providing  a  compatible  path  to  its  (later)  IBM  SPl  and  SP2  systems.  The  system, 
including  experimental  high-speed  switch,  was  used  for  production  work  by  the 
commuruty,  which  has  now  migrated  to  the  SPl  and  SP2  environments.PORTABLE 
Parallel  programming  Tools 

Although  the  architectures  of  massively  parallel  systems  differ  greatly,  the  major 
time  and  money  investment  of  the  research  community  (as  contrasted  to  the  cen- 
ter's personnel)  is  in  developing  and  converting  codes  (porting)  to  operate  in  differ- 
ent environments.  The  dose  cooperation  between  the  NSF  Centers  via  the  Meta- 
Center  and  informal  contacts  among  its  research  users  and  cooperating  agencies 
such  as  ARPA  and  the  various  National  Labs  have  resulted  in  substantial  progress 
ensuring  that  labor  intensive  programming  operations  need  not  be  duplicated  need- 
lessly. 

Prototype  Parallel  Programming  Environments 

Working  with  the  Parascope  Group  at  the  Center  for  Research  in  Parallel  Comput- 
ing, an  NSF  Sdence  and  Technology  Center,  CTC  developed  extensions  supporting 
new  parallel  programming  paradigms  and  extensions  making  ports  from  one  type  of 
parallel  programming  platform  to  another  easier.  ParaScope  is  a  prototype  parallel 
programming  environment.  Both  the  BIMA  and  Cosmology  GCs  at  NCSA  are  work- 
ing closely  with  Indiana  University  Computer  Scientist  Dennis  Gannon  to  move  ap- 
plication codes  previously  written  in  FORTRAN  to  the  portable  pC++  which  is  the 
model  for  HPC++,  the  equivalent  of  HPF  in  the  C++  world. 
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Extensions  of  PVM 

In  the  Dome  project,  Adam  Beguelin,  one  of  the  original  developers  of  PVM  now 
working  jointly  at  PSC  and  in  Computer  Science  at  Carnegie  Mellon  University,  is 
extending  PVM  to  improve  load  balancing  and  fault  tolerance.  His  work  is  guided  by 
the  experience  of  PSC's  cluster  users. 

Scalable  Libraries 

The  goal  of  the  ARPA  funded  Scalable  Parallel  Libraries  project  is  to  develop  math- 
ematical software  libraries  for  massively  parallel  processors  that  are  roughly  compa- 
rable in  scope  to  the  math  libraries  typically  available  on  conventional  supercomput- 
ers. Michael  Heath  and  his  group  at  NCSA  are  one  team  in  this  multi-institutional 
project  and  they  have  been  developing  parallel  direct  methods  for  solving  sparse  lin- 
ear systems.  For  this  purpose,  they  have  developed  a  fully  parallel  sparse  solver  for 
distributed  memory  parallel  computers.  Unlike  most  other  efforts,  which  have  fo- 
cused only  on  factorization,  this  solver  performs  all  phases  of  the  computation  in 
parallel  mode,  including  the  symbolic  preprocessing  necessary  to  reorder  the  sparse 
matrix  and  distribute  it  across  processors  to  maintain  data  locality.  With  funding 
from  IBM,  CTC  staff  developed  scalable  versions  of  key  numerical  library  routines 
for  its  IBM  cluster  system;  these  routines  were  included  by  IBM  in  its  ESSL  library 
product. 

Storage  Technologies 

With  the  vast  increase  in  both  simulation  and  observational  data,  the  MetaCenter 
has  worked  a  great  deal  on  problems  of  storage  technologies.  Here  again,  many  of 
the  biggest  areas  of  progress  are  in  software.  The  creation  of  a  universal  file  format 
standard,  a  national  file  system  with  a  single  name  space,  and  a  multivendor  archiv- 
ing software  are  some  of  the  results  of  MetaCenter  innovation,  collaboration  with 
computer  scientists,  and  with  other  leading  national  laboratories.  There  are  even 
examples  of  the  Program's  computational  facilities  being  used  to  improve  the  basic 
storage  capacity  of  the  physical  medium  of  storage  itself. 

AFS-Establishing  a  National  File  System 

PSC  recognized  that  the  Andrew  File  System  (AFS),  developed  at  Carnegie  Mellon 
University  with  IBM  support  for  a  workstation  environment,  was  particularly-well 
suited  for  use  in  high  performance  computing,  because  of  its  superior  security,  scal- 
ing properties,  and  manageability.  PSC  undertook  a  major  program  of  adapting  AFS 
to  the  high  performance  computing  environment  which  has  led  to  a  MetaCenter 
wide  effort  to  develop  a  shared  national  file  system.  PSC's  AFS  enhancements  to 
Cray's  UNICOS  are  installed  at  a  number  of  advanced  computing  centers  (SDSC, 
NERSC,  rPP,  LRZ  (Germany),  ETH  (Switzerland)  and  the  University  of  Stuttgart). 
PSC  has  also  extended  AFS  to  multi-resident  AFS  which  enables  AFS  to  be  a  com- 
ponent of  a  hierarchical  storage  system,  permitting  transparent  and  cost-effective 
storage  of  large  amounts  of  data.  These  enhancement  are  installed  at  other  Meta- 
Center sites,  NERSC,  Cray  Research,  Transarc  Corporation,  Max  Planck  Institute  for 
Plasma  Physics,  and  the  University  of  Cologne.  With  AFS,  distributed  applications 
across  the  MetaCenter  are  now  possible.  For  instance,  Paul  Dawson  (Dept.  of  Mech. 
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and  Aerospace  Engineering,  Cornell  Univ.),  simulating  deformations  of  aluminum, 
uses  AFS  to  build  a  distributed  application  with  modeling  performed  at  PSC  and  vi- 
sualization at  the  CTC.  The  SDSC  is  running  an  AFS  cell  supporting  the  Computa- 
tional Center  for  Macromolecular  Structure  (CCMS),  an  SDSC,  UCSD,  and  Scripps 
Research  Institute  collaboration.  The  charter  of  the  CCMS  is  the  development  and 
distribution  of  portable,  innovative  software  for  the  study  of  macromolecular  struc- 
ture. AFS  simplifies  the  cross-institution  distribution  and  maintenance  of  software 
and  textual  information. 

CTC  was  the  first  Center  to  put  AFS  in  production  on  its  HPC  systems;  in  fact,  all 
CTC  production  systems,  except  KSR,  and  servers  are  integrated  through  AFS,  in- 
cluding its  new  mass  storage  environment.  CTC  has  lead  in  a  number  of  areas  of  file 
system  integration,  including  a  project  with  the  University  of  Michigan  integrating 
its  AFS  mainframe  port,  IFS,  into  its  HPC  environment,  and  joint  efforts  with 
TRANSARC  in  improving  AFS  performance  over  high  speed  networks,  including 
an  FDDI  testbed.  New  efforts  involving  TRANSARC  include  optimization  over 
ATM  and  within  IBM's  SP2  scalable  switch. 

HDF-Creating  a  Standard  File  Format 

The  NCSA  Hierarchical  Data  Format  (HDF),  created  by  Michael  Folk  and  his  group  at 
NCSA  has  become  one  of  the  leading  self-describing  file  formats  in  the  world  today. 
Many  scientific  institutions,  organizations  and  programs  have  adopted  HDF  as  a 
standard  file  format  for  data  exchange  and/or  archiving.  In  1992,  NASA  selected  HDF 
as  the  basis  from  which  to  develop  an  EOSDIS  standard  data  format  (SDF).  The  goal 
of  SDF  is  to  provide  a  single,  self-describing  format  for  distributing  data  derived  from 
approximately  1.2  terabytes  of  data  daily  that  EOSDIS  will  eventually  produce.  Other 
examples  of  HDF  adoption  include  the  Institute  of  Applied  GeoScience  (seismic 
data).  Pacific  Northwest  Laboratory  (cancer  research).  Children's  Hospital  in  Boston 
(x-ray  images),  the  UCLA  Scientific  Visualization  Lab.  By  working  closely  with  many 
different  user  communities  to  support  the  harmonization  of  data  models  and  meta- 
data conventions  across  as  many  disciplines  as  possible,  NCSA  is  helping  to  create  a 
software  foundation  for  the  Nil  enabling  it  to  reach  its  potential  to  support  the  broad- 
est constituency  possible. 

Migrating  to  a  Standard  Archiving  Software 

In  1985,  the  NSF  Centers'  goal  was  simply  to  establish  national  access  for  the  aca- 
demic community  to  the  type  of  advanced  supercomputing  and  archiving  systems 
found  in  the  Dept.  of  Energy  national  laboratories.  NCSA  and  SDSC  duplicated  the 
Los  Alamos  computing  environment  in  1985,  including  the  Common  File  System 
(CFS)  archive,  while  PSC  adopted  Westinghouse's  PDM  software  that  had  additional 
data  migration  facilities.  As  time  went  on,  the  Centers  began  to  develop  innovation 
in  storage  software.  It  is  the  strategy  of  the  MetaCenter  to  explore  alternative  techni- 
cal approaches  to  storage  at  the  bits  level,  while  maintaining  interoperability 
through  standard  protocols.  SDSC,  through  the  DISCOS  (Distributed  Computer  So- 
lutions) division  of  General  Atomics  (which  is  now  owned  by  OpenVision),  pio- 
neered productization  of  distributed,  hierarchical  file  and  storage  management  ap- 
plications software  for  networked,  multi-vendor  and  open  systems  environments, 
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based  on  the  IEEE  storage  model.  Working  with  the  UIUC  Computer  Science  De- 
partment, NCSA  was  able  to  encode  data  migration  and  caching  strategies  into  CFS 
to  improve  its  ability  to  minimize  disk  cache  misses.  In  1992,  NCSA  developed  CFS- 
to-UniTree  data  formatting  and  migration  tools  as  well  as  a  suite  of  archive  man- 
agement tools.  PSC  integrated  Cray's  proven  commercial  archiving  technology 
(DMF)  with  more  usable  front-end  software  and  with  its  multi-resident  APS.  SDSC 
collaborates  with  OpenVision  and  the  National  Storage  Laboratory  in  the  further 
development  and  stabilization  of  UniTree  as  a  robust  production  archival  storage 
system.  SDSC  has  developed  transaction  journaUng  software  that  is  critically  impor- 
tant for  guaranteeing  integrity  of  the  file  nameserver.  CTC  and  SDSC  are  installing 
the  first  generation  of  a  high-performance  variant  of  UNITREE  which  has  been  de- 
veloped by  the  National  Storage  Laboratory,  to  provide  enhanced  I/O  capability  to 
balance  the  increase  in  installed  computing  capacity.  CTC  will  continue  to  work 
closely  with  IBM  in  the  testing  and  deployment  of  the  next  generation  of  mass  stor- 
age systems,  the  High  Performance  Storage  System  (HPSS).  This  will  include  the  ca- 
pability of  utilizing  parallel  I/O  to  speed  data  transfer  to  the  SP2. 

Development  of  high-density  magnetic  media 

The  Center  for  Magnetic  Recording  Research  (CMRR),  located  on  the  campus  of 
UCSD,  has  funding  from  NSF  and  21  corporations  having  enterprises  connected 
with  the  storage  and  retrieval  of  magnetically  written  information.  The  chief  tech- 
nical problem  CMRR  has  been  addressing  has  been  magnetic  noise  in  the  metallic 
thin-film  media  used  to  coat  high-density  disks.  Neal  Bertram  (Dept.  of  Electrical 
and  Computer  Engineering,  UCSD)  is  a  researcher  at  CMRR  who  tackles  the  prob- 
lem of  magnetic  noise  computationally.  Magnetic  thin  films  are  polycrystalline, 
rather  than  continuous  or  amorphous,  and  the  grainy,  particulate  nature  of  the 
medium  is  a  fundamental  source  of  noise.  Bertram's  calculations  have  explored  the 
effects  of  two  primary  types  of  interaction  between  grains  that  cause  noise:  magneto- 
static  coupling  and  exchange  coupling.  The  calculations  resulted  in  recommenda- 
tions for  alloys  and  fabrication  processes  that  would  reduce  noise  from  both  sources. 
The  research  enabled  engineers  at  IBM's  Almaden  Research  Center  to  design  a  disk 
coating  that  packs  a  gigabit  (one  billion  bits)  of  information  onto  a  square  inch, 
which  is  15-30  times  current  storage  densities.  Bertram  has  now  turned  his  atten- 
tion to  calculations  of  other  effects,  including  giant  magnetoresistance,  that  can  be 
important  in  designing  high-density  disk  media  and  disk  recording  and  playback 
heads.  His  codes  model  the  process  of  recording  bits  in  intricate  detail;  the  process  of 
laying  down  a  single  bit  of  information  takes  several  minutes  to  calculate  on  SDSC's 
Cray  Research  C90. 

Networking 

One  of  the  great  successes  of  the  NSF  MetaCenter  has  been  in  providing  the  "high- 
end  pull"  that  has  led  to  the  creation  and  exponential  evolution  of  the  NSFnet.  As  a 
result,  the  NSFnet  backbone  of  1995  has  3000  times  the  bandwidth  of  the  backbone  of 
1986.  The  Centers  have  also  prototyped  the  high  performance  local  area  networks 
that  are  needed  to  feed  into  the  national  backbone  as  well  as  the  next  generation  of 
gigabit  backbones.  Security  over  networks  is  essential  not  only  for  industrial  usage. 
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but  more  and  more  for  widespread  citizen  usage.  Again,  the  MetaCenter  has  created 
innovations  such  as  dealing  with  industrial  firewalls.  The  existence  of  the  MetaCen- 
ter network  testbeds  allows  for  new  kinds  of  science  to  be  attacked,  perhaps  best  il- 
lustrated by  the  rise  of  telemicroscopy,  in  which  leading  edge  projects  are  being  car- 
ried out  at  each  of  the  NSF  Supercomputer  Centers.  With  the  national  priority  on 
the  Nn,  the  Centers  are  moving  rapidly  to  expand  their  networking  research  to 
community  based  Nil  testbeds,  including  local  healthcare,  education,  government, 
and  small  business  partners. 

Evolution  of  NSFNET 

The  56kbps  connection  between  the  NSF  Centers,  established  in  1986,  was  the  begin- 
ning of  the  NSFnet.  Based  on  the  successes  of  ARPAnet  and  the  TCP/IP  protocol 
within  the  computer  science  and  Dept.  of  Defense  communities,  the  NSFnet  rapidly 
grew  to  provide  remote  access  to  the  NSF  Supercomputer  Centers  by  the  creation  of 
regional  and  campus  connections  to  the  backbone.  Although  started  by  the  pull  from 
the  high  end,  the  NSFnet  soon  began  to  provide  ubiquitous  connectivity  to  the  aca- 
demic research  community  for  electronic  mail,  file  transport,  and  remote  login,  as 
well  as  supercomputer  connectivity.  These  daily  uses  soon  became  indispensable  to 
the  research  community  and  the  sustained  exponential  growth  of  the  Internet  took 
off.  The  MetaCenter's  industrial  partner  network  was  among  the  first  in  the  corpo- 
rate world  to  use  NSFnet /Internet  technology  to  connect  corporations  to  the  Internet 
for  the  purposes  of  computational  science.  This  was  an  important  precursor  for  to- 
day's rapid  commercialization  of  the  Internet. 

By  early  1995,  the  NSFnet  will  return  to  a  high  speed  backbone  connecting  the  Meta- 
Center and  some  of  the  newly  selected  MetaCenter  Regional  Alliance  members. 
However,  the  bandwidth  of  the  backbone  will  be  3000  times  higher  than  that  of  the 
original  backbone  (56  kbps).  While  some  tend  to  think  of  the  MetaCenter  as  focusing 
on  high  performance  computing  only,  it  is  useful  to  remember  that  computing 
power  of  the  fastest  supercomputer  processor  in  the  program  has  grown  by  little 
more  than  100  times  during  the  same  period.  Indeed,  it  is  likely  that  the  155  mbps 
vBNS  will  be  upgraded  to  622  mbps  within  two  years.  Even  by  the  time  the  Centers 
receive  the  first  teraflop  machines  in  1997-98,  realizing  a  factor  of  1000  increase  in 
speed  over  1985,  the  backbone  will  have  grown  by  a  factor  of  at  least  25,000  fold  in 
bandwidth. 

As  part  of  an  SDSC/UCSD  collaboration,  Kimberly  Claffy  recently  completed  a  Ph.D. 
dissertation  that  outlined  a  methodology  for  profiling  Internet  traffic  flows  at  a  vari- 
ety of  granularities.  The  methodologies  and  models  developed  as  part  of  this  traffic 
characterization  effort  should  prove  very  useful  as  the  Internet  evolves  to  an  even 
larger  system  in  which  the  traffic  composition  needs  to  be  understood,  particularly 
for  planning  future  technology  and  capacity. 

High  Performance  LANs 

The  center's  program  has  also  pioneered  several  transitions  in  local  area  and 
metropolitan  area  networks  both  on  site  and  on  university  campuses,  acting  as  a  pro- 
totyping facility  for  other  campuses  who  needed  to  know  how  to  develop  long  range 
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networking  plans  for  their  campuses.  In  1988,  NCSA  installed  the  first  Ultranet  Giga- 
bit LAN  networks  with  multiple  supercomputers  and  demonstrated  480  Mbit/s  be- 
tween the  CRAY-2  and  Cray  Y-MP  supercomputers.  In  1989,  NCSA  replaced  tradi- 
tional HYPERchannel  backbone  networks  with  the  then-emerging  100  Mbit/s  FDDI 
standard.  In  1991,  PSC  began  its  move  to  a  HIPPI-based  interconnect  between  its  ma- 
jor systems. 

In  1993,  NCSA,  several  industrial  partners,  and  the  UIUC  Computer  Science  De- 
partment established  a  local  area  ATM  network  testbed  to  help  corporations  gain 
hands-on  experience  with  ATM  switches  and  interfaces.  Insight  from  this  ATM 
testbed  has  already  been  used  to  develop  long-range  corporate  network  strategies  for 
J.P.  Morgan,  Phillips  Petroleum,  FMC  Corporation,  and  United  Technologies. 

CTC  was  the  first  site  to  integrate  ATM  into  a  parallel  supercomputer  environment 
on  its  IBM  SPl  in  April  1994  and  its  IBM  SP2  in  July.  ATM  will  be  used  for  AFS-based 
file  service  and  other  high  speed  transport  needs,  including  distributed  applications 
and  image  transport. 

Gigabit  Testbeds 

Since  1987,  NCSA  and  the  Uruversity  of  Illinois  Computer  Science  Department  have 
worked  with  AT&T  on  the  XUNET  research  network  testbed  with  capacity  that  is  one 
step  beyond  what  is  available  on  the  Internet.  While  the  NSFnet  has  moved  from  56 
Kbs  through  1.5  Mbps  to  45  Mbs,  XUNET  has  moved  from  1.5  Mbs  through  45  Mbs  to 
622  Mbs.  In  July  1993,  XUNET  was  upgraded  to  622  Mbs,  the  first  network  testbed  to 
interconnect  ATM  switches  using  622  Mbs  transmission  technology  over  long  (>50 
miles)  distance  using  pure  optical  fibers  with  in-line  optical  amplifiers. 

PSC  has  worked  with  CMU's  Computer  Science  Department  in  the  Nectar  Metropoli- 
tan area  gigabit  testbed  to  develop  new  networking  technology  for  very  high-speed, 
low-latency  multi-machine  interconnects  and  to  develop  the  applications  base  which 
can  benefit  from  such  technology.  This  work  is  fully  collaborative  with  PSC's 
ground-breaking  systems  and  applications  level  work  in  heterogeneous  systems.  As  a 
result  of  testbed  work,  numerous  applications  can  now  run  routinely  between  ad- 
vanced machines  at  PSC's  main  hardware  facility  and  those  on  the  CMU  campus,  15 
miles  distant,  at  speeds  of  up  to  1  Gbit/s. 

In  partnership  with  NYNEX,  Syracuse  and  Rome  Laboratory,  now  extended  to 
Columbia,  SUNY  Stonybrook  and  Polytechnic  Institute,  CTC  participated  in  building 
a  production-level  ATM  network  focused  on  demonstrating  research  and  commer- 
cial applications.  This  network  was  demonstrated  to  the  Governor  of  New  York  in 
January  1994.  NYNET  is  also  a  Nil  testbed,  involving  outreach,  medical  applications, 
video  on  demand,  as  described  in  a  later  section. 

Various  applications  are  being  tested  in  the  CASA  testbed,  in  which  SDSC  is  a  major 
partner.  Besides  the  chemical  reaction  dynamics,  led  by  Aron  Kuppermarm  of  Cal- 
tech  and  mentioned  above  in  the  section  on  superlinear  speedup,  there  is  a  coupled 
atmosphere /ocean  model  developed  by  the  group  led  by  Roberto  Mechoso  at  UCLA. 
Another  is  Calcrust,  a  project  directed  by  JPL,  which  has  used  distributed  heteroge- 
neous computing  over  the  CASA  links  to  combine  satellite  imaging,  seismic  data. 
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and  surface  topography  in  visualizing  the  foci  of  aftershocks  of  the  1992  Landers 
earthquake  in  Southern  California. 

Secure  Networks 

In  1987  NCSA  installed  a  1.5  Mbit/s  DS-1  connection  to  Eastman  Kodak  in  Rochester, 
New  York,  followed  by  another  DS-1  connection  to  Amoco  laboratories  in  Chicago 
and  Tulsa.  By  1990  NCSA  had  connected  over  a  dozen  industrial  laboratories  to  the 
Internet  using  a  combination  of  innovative  security  precautions.  These  included 
various  forms  of  "firewalls"  which  have  now  become  commonplace  on  the  Internet. 

New  Science  Enabled  by  Networks— Telemicroscopy 

The  San  Diego  Microscopy  and  Imaging  Resource  (SDMIR),  led  by  UCSD  neurocy- 
tologist  Mark  Ellisman,  is  an  NIH-funded  Research  Resource  centered  on  a  new, 
fully  computerized  Intermediate  Voltage  Electron  Microscope  (IVEM).  The  FVEM  is 
used  to  look  at  comparatively  thick  tissue  sections  (2-10  microns)  and  it  has  been 
employed  in  studies  of  cortical  neurons  with  and  without  symptoms  of 
Alzheimer's,  in  studies  of  another  type  of  brain  cell,  called  a  Purkinje  neuron,  and 
studies  of  cell  membranes.  A  long-term  collaboration  between  SDMIR  and  SDSC  has 
made  the  microscope  usable  interactively,  over  the  Internet,  coupled  to  the  SDSC 
Cray  Research  C-90. 

Computational  analysis  and  simulation  is  allowing  biomedical  researchers  to  study 
and  predict  the  activity  of  potential  new  drugs  at  the  molecular  level.  CTC  is  working 
jointly  with  Steven  Ealick,  et  al.,  director  of  MacCHESS,  a  group  using  the  Cornell 
High  Energy  Synchrotron  Source  for  Macromolecular  Modeling.  Using  existing  high- 
speed connections  between  CTC  and  MacCHESS,  the  project  is  building  the  capability, 
for  the  first  time,  for  pharmaceutical  companies  and  academic  researchers  to  interact 
dynamically  with  x-ray  crystallographic  analyses  at  the  synchrotron,  rather  than  dis- 
covering long  after  the  beam  run  that  the  sample  was  defective  or  the  beam  position- 
ing non-optimal.  While  initially  the  researchers  are  using  processors  on  the  CTC  SP 
systems,  ultimately  a  small  IBM  SP  may  be  installed  at  the  synchrotron  site  used  for 
dynamic  analysis,  with  longer-scale  simulation  needs  being  met  using  the  far  larger 
SP2  at  tfie  CTC. 

PSC  is  working  with  the  Center  for  Light  Microscopy  and  Biotechnology,  an  NSF  Sci- 
ence and  Technology  Center  at  Carnegie  Mellon  University,  to  develop  an  Auto- 
mated Interactive  Microscope.  This  microscope  will  couple  leading  edge-microscopy 
and  high  performance  computing  through  high  speed  networks  allowing  the  real- 
time tagging  of  chemical  reactants  in  the  cell.  It  will  open  new  research  horizons  in 
biology  by  giving  researchers  the  ability  to  control  the  release  of  chemically  active 
agents  at  critical  moments  in  cell  life,  and  to  monitor  the  cell's  subsequent  develop- 
ment. 

The  personal  computer  controlling  a  scanning  tunneling  microscope  (STM)  in  the 
Beckman  Institute  at  UIUC  used  software  integrated  over  a  LAN  with  the  NCSA 
Convex  C3880,  TMC  CM-5,  and  SGI  graphics  workstation  to  enable  realtime  imaging 
and  nanolithography  of  silicon  surfaces  in  order  to  create  novel  quantum  electronic 
devices.  Working  with  the  laboratory's  director,  Joseph  Lyding  (Dept.  of  Electrical  and 
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Computing  Engineering,  UIUC  and  Beckman  Institute)  and  inventor  of  a  widely 
used  STM,  NCSA  staff  members  Rachael  Brady  and  Clint  Potter  extended  this 
telemicroscope  to  the  Internet  and  demonstrated  the  feasibility  of  using  advanced 
imaging  instrumentation  linked  with  advanced  computing  capabilities  from  any- 
where in  the  world.  The  project  was  featured  in  the  special  issue  of  Research  and 
Development  Magazine  (Oct  25,  1993)  on  Winning  in  the  21st  century. 

Nil  Testbeds 

As  a  partner  in  Common  Knowledge:  Pittsburgh,  an  innovative  project  introducing 
networking  and  computing  into  the  entire  Pittsburgh  School  District,  PSC  is  working 
with  numerous  partners,  including  Digital  Equipment,  Apple  Computer  and  both 
telephone  (Bell  of  Pennsylvarua)  and  cable  TV  (TCI)  companies,  to  create  a  prototypi- 
cal, cost-effective  approach  to  widespread  use  of  advanced  technology  in  public  edu- 
cation. 

In  collaboration  with  researchers  and  physicians  at  the  Uruversity  of  Pittsburgh  Med- 
ical Center,  the  PSC  is  developing  an  Nil-based  digital  library  of  pathology  images, 
and  the  applications  and  software  technology  which  wiU  enhance  the  practice,  teach- 
ing and  cost-effective  delivery  of  pathology. 

NCSA  in  collaboration  with  UIUC  and  the  Champaign  County  Chamber  of  Com- 
merce have  been  building  CCnet,  an  Nil  testbed  during  the  last  18  months.  Over  200 
people  from  over  70  community  organizations  have  been  involved  since  April,  1993 
in  defining  six  major  applications  experiments  in  small  business,  health  care,  educa- 
tion, government  and  community  services,  agribusiness,  and  geographic  informa- 
tion systems  (GIS).  The  first  of  20  multimegabit/s  links  into  the  community  was  es- 
tablished with  the  Urbana  Free  Library  in  August  1994.  All  the  high  schools  and  a 
number  of  small  businesses  are  hooking  on  in  September.  NCSA  is  establishing  a 
large  GIS  server  which  will  be  available  over  CCnet  to  community  projects.  Partners 
in  CCnet  include  Time-Warner  cable,  Ameritech,  DEC,  Motorola,  and  potentially 
MCI  and  AT&T. 

NYNET,  one  of  the  gigabit  testbeds,  is  also  designated  as  an  NE  testbed  providing 
outreach,  medical  applications,  video  on  demand  to  CTC  and  New  York  academic 
and  industrial  partners. 

InterNIC  is  the  latest  in  an  evolutionary  line  of  support  from  the  NSF  for  the  use  of 
the  Internet  by  the  science,  research,  and  education  communities.  The  InterNIC  pro- 
vides three  types  of  services:  Information  Services  (provided  through  General  Atom- 
ics and  the  SDSC),  Directory  and  Database  Services,  and  Registration  Services.  Infor- 
mation Services  provides  procedures  for  connecting  to  the  Internet,  pointers  to  re- 
sources and  tools  available  over  the  network,  training  seminars  for  new  and  experi- 
enced users  and  up-to-date  reports  on  new  resources  and  activities  on  the  Internet. 
Several  irmovative  approaches  to  distributed  services  have  been  implemented,  in- 
cluding the  InfoGuide,  an  on-line  Internet  information  service.  The  Scout  Report  is 
a  weekly  summary  of  Internet  highlights  which  combines  in  one  place  the  highlights 
of  new  resource  announcements  and  other  news  that  occurred  on  the  Internet  dur- 
ing the  previous  week.  The  InterNIC  Reference  Desk  acts  as  the  "NIC  of  first  and  last 
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resort."  The  desk  supports  a  variety  of  users  answering  "starter"  questions  from 
novice  users  who  are  unfamiliar  with  the  Internet  as  well  as  specialized  questions 
from  intermediate  and  advanced  users. 

VISUALIZATION  AND  VIRTUAL  REALITY 

The  NSF  center's  were  instrumental  in  bringing  the  notion  and  tools  of  scientific  vi- 
sualization to  the  research  commuruty  in  the  1980s.  By  combining  advanced  visual- 
ization resources  with  simulation  datasets  created  by  remote  users  on  the  centers 
program's  high  performance  computers,  new  visualization  paradigms  for  interpret- 
ing numerical  data  were  developed.  This  led  scientists  to  consider  visualization  as  an 
intimate  part  of  their  computational  toolkit.  In  addition,  the  centers  worked  closely 
with  the  pre-existing  computer  graphics  community  to  get  them  creating  new  tools 
for  scientists  as  well  as  for  entertainment.  Already  by  1987,  the  staffs  of  the  centers, 
working  with  national  users,  were  creating  scientific  visualizations  so  compelling 
that  they  became  regularly  chosen  to  be  part  of  the  SIGGRAPH  Film  and  Video 
Show,  the  "academy  awards"  of  the  visualization  industry.  Today  the  centers  visual- 
ization staff  and  their  allied  visualization  Centers  are  at  the  forefront  of  research  into 
how  to  turn  virtual  reality  technologies  into  useful  tools  for  scientific  and  engineer- 
ing research. 

Development  of  Scientific  Visualization 

From  its  inception,  NCSA  has  worked  with  computer  artists  like  Donna  Cox  (UIUC 
Dept.  of  Art)  and  Dan  Sandin  (UIC  School  of  Art  and  Design)  and  computer  scientists 
like  Tom  DeFanti  (Dept.  of  Electrical  Engineering  and  Computer  Science,  UIC)  to  cre- 
ate cross  disciplinary  teams  with  end  users  in  order  to  create  new  levels  of  scientific 
visualizations.  NCSA  also  hired  a  number  of  staff  from  leading  companies  in  the 
California  entertainment  industry  to  bring  the  software  tools  of  special  effects  in 
movies  or  TV  commercials  to  the  use  of  the  scientific  and  engineering  communities. 
Initially,  the  NCSA  visualization  environment  was  built  on  Alliant  shared-memory 
multiprocessors  using  the  Wavefront  visualization  software.  As  specialized  graphics 
hardware  became  available  on  Silicon  Graphics  systems,  the  NCSA  visualization 
environment  migrated  from  the  Alliant  to  Silicon  Graphics  systems.  The  scientific 
visualizations  created  by  NCSA  s*^'ff  have  not  only  broken  new  ground  for  scientists 
viewing  their  data,  they  have  also  won  awards  worldwide  for  aesthetic  quality.  NC- 
SA's Renaissance  Experimental  Laboratory  (REL),  created  by  Donna  Cox  with  a  major 
donation  from  Jim  Clark  (founder  of  Silicon  Graphics)  was  the  first  advanced  visual- 
ization training  facility  in  the  centers  and  continues  to  support  university  courses  in 
geology,  mathematics,  graphics  design,  computer  science,  and  other  disciplines. 

The  SDSC  has  developed  a  variety  of  software  tools  that  can  be  used  to  access  and 
connect  existing  visualization  resources  automatically.  The  goal  has  been  to  provide 
researchers  with  training  and  access  to  tools  that  will  support  their  research  needs. 
The  tools  include:  1)  The  SDSC  Image  Library,  a  collection  of  image  manipulation 
and  conversion  utility  routines  that  can  be  embedded  in  existing  software.  An  inter- 
esting example  of  the  use  of  this  library  are  the  image  conversion  modules  devel- 
oped by  the  International  AVS  Center,  which  quickly  became  the  top  2  user  modules 
in  their  distribution;  2)  the  SDSC  Image  Tools,  a  collection  of  utilities  based  on  the 
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Image  Library,  are  software  tools  for  reading,  writing,  and  manipulating  raster  im- 
ages. This  toolset  also  allows  researchers  to  convert  file  formats  among  over  thirty 
widely  used  graphics  formats  (e.g.,  from  HDF  to  PICT). Now  in  its  second  release,  it 
runs  on  Cray  Research,  DEC,  HP,  IBM,  SGI,  and  Sun  Microsystems  platforms.  Over 
4,000  sites  worldwide  have  uploaded  the  Image  Tools  from  SDSC's  anonymous  FTP 
area;  3)  vpr,  a  client-server  visualization  hardcopy  utility,  vpr  takes  advantage  of  the 
Internet  by  allowing  remote  users  to  send  images  to  local  hardcopy  devices.  Popular 
hardcopy  devices  that  are  connected  to  vpr  include  a  variety  of  film  recording  de- 
vices, a  color  paper  plotter,  and  video  recording;  and  4)  the  SDSC  Color  Tutorial,  a 
SuperCard-based  hypertext  exploration  in  color  theory  for  computer  graphics.  Exam- 
ples show  the  different  points  that  are  being  made,  while  hyperlinks  allow  the  user 
to  jump  to  the  most  interesting  references. 

CTC  Visualization  staff  developed  Visual  Programming  Language  for  Animation 
(VPLA),  a  program  that  easily  integrates  sound  and  image  sequences  into  scientific 
animations.  VPLA  can  use  rendered  images  from  several  standard  visualization 
packages  including  DataExplorer.  As  the  national  repository  for  Data  Explorer  soft- 
ware and  lead  training  site,  the  CTC  works  with  faculty,  industrial  users  and  stu- 
dents across  the  country  in  developing  state-of-the  art  animations  and  images.  In 
addition,  using  DX,  the  CTC  has  spearheaded  using  visual  programming  languages 
not  only  for  visualizing,  but  for  managing  distributed  applications  running  across 
the  centers  program.  For  example,  the  CTC  built  a  Data  Explorer  module  allowing  its 
researchers  to  access  the  CMS  at  NCSA.  The  CTC  was  instrumental  in  IBM's  agree- 
ing to  support  DX  across  all  major  vendor  platforms,  including  SGI. 

PSC  has  concentrated  its  visualization  efforts  on  the  development  of  tools  for  re- 
mote users.  Its  GPLOT  software  is  in  use  at  over  300  sites.  Its  automated  animation 
facility  has  enabled  researchers  to  produce  hundreds  of  videotapes  without  physi- 
cally visiting  the  any  specific  center.  It  is  now  turning  its  efforts  to  develop  such 
tools  embodying  virtual  reality. 

Virtual  Reality  Impacts  Industrial  Design 

In  1992  NCSA  began  a  transition  from  the  now-traditional  workstation  visualiza- 
tion activities  to  virtual  environments,  with  leadership  provided  by  Caterpillar  Inc., 
an  NCSA  Industrial  Partner.  Traditionally,  translating  electronic  CAD  blueprints 
into  full  scale  wooden  models  of  new  heavy  earth  moving  equipment  in  order  to 
evaluate  design  changes  required  6-9  months.  Working  with  NCSA  staff.  Caterpillar 
built  up  a  VR  laboratory  in  the  UIUC  Beckman  Institute  and  networked  the  SGI 
graphics  workstations  which  create  the  VR  images  to  their  Peoria  headquarters  facil- 
ities. Using  a  variety  of  VR  viewing  technologies,  a  number  of  design  options  al- 
ready have  been  tested  for  new  models  of  Caterpillar  wheel  loaders  and  backhoe 
loaders  that  will  be  introduced  by  1996.  Design  changes  can  now  be  made  in  less  than 
one  month.  Caterpillar  design  engineers  Dave  Stevenson  and  John  Bettner  received 
the  1993  NCSA  Industrial  Grand  Challenge  Award  for  their  innovative  work.  Media 
coverage  of  this  award  reached  over  200  media  outlets. 
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Development  of  Immersive  Science  Projects 

The  transition  from  stand-along  workstation  visualization  to  Nil  distributed  visual- 
ization was  emphasized  at  SIGGRAPH'92  in  Chicago  when  NCSA  collaborated  with 
the  Ul-Chicago's  Electronic  Visualization  Laboratory  and  dozens  of  science  teams  to 
demonstrate  wide  area  interactive  visualization  at  Showcase.  A  new  level  of  real- 
ism in  virtual  reality  was  debuted  there  as  well  with  the  public  showing  of  EVL's 
Cave  Automated  Virtual  Environment  (CAVE),  which  provided  complete  immer- 
sion in  complex  3-D  data  sets  at  workstation  levels  of  resolution.  In  1993-1994,  EVL, 
NCSA,  and  Argonne  organized  a  national  call  for  proposals  which  resulted  in  over 
60  EVL/NCSA  computer  science  and  visualization  staff  and  graduate  students  help- 
ing researchers  from  over  30  institutions  in  porting  their  applications  into  the 
CAVE  environment.  For  the  first  time,  this  included  realtime  coupling  to  parallel 
supercomputers  so  that  dynamic  3-D  evolutions  could  be  viewed  immersively  and 
steered  interactively.  At  SIGGRAPH  94,  8,000  attendees  were  able  to  directly  experi- 
ence these  science  projects.  CTC  developed  a  specific  Cave  Visualization  on  Macro- 
molecular  Modeling:  the  Structure  of  Acetylcholine  Esterase;  this  visualization  was 
integral  to  the  researcher's  understanding  of  the  molecule's  activity.  This  applica- 
tion runs  not  only  on  the  CAVE's  SGI  workstations,  but  on  CTC's  IBM  SP2  system  as 
well. 

Virtual  Reality  over  ATM  networks 

In  a  project  IN  1994  with  Rome  Laboratory,  demonstrated  virtual  reality  techniques 
over  an  ATM  network  between  CTC  SGI  computers  and  Rome  Laboratory.  Research- 
ing ATM  technologies  for  real  time  applications  and  demonstrating  software  tools 
for  application  steering  important  to  molecular  modeling,  telemedicine  and  com- 
mand and  control. 

The  Sequoia  2000  Visualization  Group  at  SDSC  developed  a  prototype  data  visualiza- 
tion system  "Tecate"  using  virtual  reality  technology  to  address  many  of  the  issues 
involved  in  exploring  the  informational  content  of  networked  data  servers.  Tecate 
enables  the  browsing  for  data  that  resides  in  repositories  managed  by  a  database  man- 
agement system  via  user-interaction  with  graphical  renditions  of  objects  that  repre- 
sent data  features. 

Alpha  Shapes,  Biomolecules,  and  Cosmology 

Alpha  shapes,  a  form  of  geometric  modeling  developed  by  the  1993  Waterman 
Award  winner  Herbert  Edelsbruner  (Dept.  of  Computer  Science,  UIUC)  and  NCSA 
staff  member  Ping  Fu,  focuses  on  the  formal  definition,  construction,  and  measure- 
ment of  shapes  for  any  given  point  set  in  space.  The  discrete  nature  of  the  alpha 
shape  complex  has  computational  advantages  over  any  other  known  method  which 
can  be  exploited  in  computing  surface  area  and  volume  of  a  space  filling  diagram  and 
in  localizing  and  measuring  voids.  The  latter  is  useful  in  studying  water  molecules 
residing  inside  a  protein.  NCSA  users  have  discovered  other  related  applications  of 
alpha  shapes  by  applying  them  to  such  diverse  fields  as  adaptive  grid  generation, 
medical  image  analysis,  visualizing  the  structure  of  earthquake  data,  and  the  large- 
scale  structure  of  the  universe. 
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DIGITAL  LIBRARIES  AND  INFOSERVERS 

The  National  Information  Infrastructure  requires  many  software,  computer,  and 
communications  resources  that  were  not  traditionally  thought  to  be  part  of  high 
performance  computing.  In  particular,  knowledge  organization,  location,  and  navi- 
gating tools  needed  to  be  developed.  The  NSF  Supercomputer  Center  staffs  and  their 
associated  universities  have  proven  to  be  fertile  ground  for  developing  their  new 
tools.  Perhaps  the  most  spectacular  success  has  been  NCSA  Mosaic,  which  in  less 
than  18  months  has  become  the  Internet  knowledge  browser  of  choice  by  over  a  mil- 
lion users.  The  Mosaic/World  Wide  Web  infrastructure  has  set  off  an  exponential 
growth  in  the  number  of  decentralized  authoring  of  information  servers. 

Digital  Libraries 

In  1989,  NCSA  as  part  of  its  XUNET  application  testbed  proposed  that  multimedia 
digital  libraries  would  require  gigabit  networks  in  order  to  fully  support  high  defini- 
tion imagery  and  the  coupling  of  large  data  sets  with  computing  resources  and  geo- 
graphically dispersed  researchers.  This  resulted  in  research  developments  like  DICE 
(Distributed  Collaboration  Environment).  Parallel  efforts  in  providing  researchers 
with  global  information  retrieval  and  display  capabilities  over  existing  environ- 
ments combined  collaboration  with  Bruce  Schatz  (then  at  U.  Arizona)  and  his 
Worm  Community  System,  with  internet-based  designs  of  component  client/server 
architectures,  like  the  World  Wide  Web.  These  approaches  influenced  the  design  of 
the  current  grand  challenge  digital  library  prototype  for  accessing  radio  astronomy 
images  and  data  sets.  NCSA  built  on  the  success  of  Internet  access  tools  such  as 
NCSA  Telnet,  adapted  this  modified  digital  library  paradigm  to  the  Internet  with 
NCSA  Mosaic.  Today,  the  grand  challenge  image  library  uses  NCSA  Mosaic  as  it's 
user  interface,  and  Schatz  has  joined  NCSA  to  head  the  recently  awarded 
NSF/ARPA  Digital  Libraries  project,  which  combines  a  testbed  based  on  the  compo- 
nent architecture  with  experiments  in  object-based  designs. 

Scalable  Information  Servers 

The  enormous  success  of  NCSA  Mosaic  and  CERN's  WorldWideWeb  has  resulted 
in  explosive  growth  in  the  use  of  NCSA's  WWW  server.  By  the  end  of  1993,  NC- 
SA's server  load  had  grown  beyond  the  capabilities  of  any  single  server.  This  re- 
sulted in  the  design  of  an  innovative  distributed  scalable  server  architecture  that  in- 
volved a  modification  of  the  Internet's  Domain  Name  System  software.  By  Sept. 
1994,  the  NCSA  WWW  server  was  handling  over  2  million  connections  per  week. 
NCSA's  Hewlett-Packard  workstation  cluster  based  distributed  information  server 
has  now  been  duplicated  at  many  WWW  and  FTP  sites  on  the  Internet  and  within 
corporations.  A  number  of  corporations  are  presently  working  with  NCSA  on  the 
next  generation  of  this  distributed  architecture. 

The  Rise  of  the  MosaicAVWW  Information  Infrastructure 

NCSA  developed  the  Mosaic  user  interface  software  which  provides  point-and-click 
access  to  the  diverse  information  storage  protocols  of  the  Internet,  such  as  World 
Wide  Web  (WWW),  Gopher,  FTP,  and  WAIS.  NCSA  Mosaic  establishes  the  neces- 
sary connections,  file  transmissior^,  decompression,  launch  of  viewer  programs,  and 
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screen  display  of  text,  images,  animations,  or  audio,  in  response  to  a  single  mouse 
click  from  the  user.  NCSA  Mosaic  is  available  for  Mac,  Windows,  and  Unix  comput- 
ers for  free  to  individual  users,  for  government  and  educational  use,  and  for  internal 
use  within  companies.  Monthly  download  rates  from  the  NCSA  site  alone  are  con- 
sistently over  30,000.  Although  accurate  estimates  are  difficult,  it  is  widely  felt  that 
over  a  million  copies  of  NCSA  Mosaic  are  in  use.  Further,  commercial  versions  of 
NCSA  Mosaic  are  available.  The  principal  licensee.  Spyglass,  Inc.,  has  announced  or- 
ders for  over  five  million  copies  of  their  enhanced  version,  with  projections  to 
twenty  million  copies  within  a  year.  Use  of  NCSA  Mosaic  has  increased  WWW  traf- 
fic on  the  NSF  backbone  by  over  10,000  fold  since  Jan.  1993.  Overall  WWW  traffic  in 
August  hit  1.3  Terabytes  or  8  %  of  the  total  NSFnet  backbone  traffic,  higher  than 
SMTP.  Because  of  this,  NCSA  has  become  the  second  biggest  Internet  site  in  the 
world  in  terms  of  traffic  from  its  site.  The  NCSA  Mosaic/WWW  information  infras- 
tructure is  allowing  for  an  enormous  growth  in  decentralized  authoring  of  infos- 
ervers  throughout  the  world.  In  1994,  NCSA  was  given  Infoworld's  Publisher's  In- 
dustry Achievement  Award. 

DESKTOP  Software 

From  the  begirming,  the  NSF  Supercomputer  Centers  provided  focal  points  for 
pulling  together  teams  of  computer  scientists  and  software  developers.  Since  the 
history  of  the  centers  has  greatly  overlapped  with  the  worldwide  rise  of  the  personal 
computer  and  workstation,  it  is  not  surprising  that  the  software  developers  focused 
on  creating  easy-to-use  software  tools  for  the  desktop  machines  themselves.  These 
tools  have  had  a  major  influence  on  the  usefulness  of  the  supercomputer  facilities 
to  the  remote  science  and  engineering  community.  The  collaboration  tools  will 
have  a  great  impact  on  tying  together  the  newly  emerging  electronic  teams  of  scien- 
tists made  possible  by  the  growth  of  the  Internet. 

Connectivity  Tools 

NCSA  Telnet  was  a  ground-breaking  desktop  application  that  provided  access  to  the 
emerging  NSF  Supercomputing  Centers  in  the  late  '80s.  Developed  by  the  new 
Workstation  Tools  Group  (later  the  SDG)  at  NCSA,  this  brought  full  TCP  connectiv- 
ity to  researchers  using  IBM  and  Macintosh  systems,  sigruficantly  broadening  the  par- 
ticipation base  beyond  Unix  users,  thereby  introducing  thousands  to  both  the  inter- 
net and  the  NSF  Centers  Program.  Continuously  supported  up  to  the  present  time, 
these  tools  have  also  led  to  a  spin-off  company  Intercon,  headed  by  one  of  NCSA 
Telnet  developers. 

Collaboration  Tools 

NCSA  has  supported  a  program  of  research  and  development  on  collaboration  tech- 
nology for  science  and  engineering  researchers  for  over  3  years.  NCSA  Collage,  a  tool 
that  runs  across  MSWindows,  Mac,  and  XWindows  systems,  provides  the  capability 
to  carry  on  remote  digital  conferencing  sessions  between  researchers.  The  first  live 
MetaCenter  collaborative  session  using  NCSA  Collage  was  held  in  1992  .  Collage 
combines  many  of  the  features  of  NCSA's  communications  and  graphic  data  analysis 
tools.  NCSA  also  continues  to  innovate  in  asynchronous  collaboration  tools  such  as 
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asynchronous  collaboration  tools,  hence  the  interest  in  annotation  and  workgroup 
support  capabilities  in  NCSA  Mosaic.  Current  work  focuses  on  choosing  and  combin- 
ing the  best  of  these  synchronous  and  asynchronous  capabilities  in  usable  next-gen- 
eration global  collaboration  tools  for  the  scientific  and  educational  communities. 

Cornell  University's  CUSEEME  video  teleconferencing  software,  aimed  at  providing 
video  teleconferencing  on  low-end  workstations,  is  in  use  at  the  NSF  Centers  and  as 
part  of  NYNET,  NYSERNET  and  other  organizations  for  routine  use,  including 
medical  projects  between  the  CTC  and  the  Cornell  University  Medical  College  in 
NYC.  This  work,  funded  by  NSF  and  Cornell  itself,  is  freely  distributed  and  runs  on 
Mac  and  PC  platforms  using  inexpensive  video  equipment.  Several  national  and  in- 
ternational collaborations  have  successfully  utilized  this  software.  The  centers  have 
gained  experience  with  traditional  video  teleconferencing  systems,  through  its  NSF- 
funded  system.  It  is  now  looking  at  packet  video  systems  using  the  vBNS  and  other 
network  facilities.  These  systems  will  have  the  capability  of  moving  the  videocon- 
ference  from  a  set  of  specially  equipped  rooms  to  the  desktop.  An  investigation  is 
now  underway  to  develop  the  optimal  system  for  CTC's  requirements. 

Graphics  Tools 

NCSA  Image  was  the  first  scientific  visualization  tool  developed  for  the  desktop 
viewing  of  supercomputing  output  in  the  program.  It  provided  the  research  com- 
munity Mac  and  Unix  based  visualization  methods  for  analysis  of  huge  data  sets,  as 
well  as  creating  some  of  the  first  client/ server  tools  which  integrated  remote  desktop 
workstations  and  personal  computers  with  the  center  programs  high  performance 
engines. 

The  SDSC  Image  Tools  are  software  tools  for  reading,  writing,  and  manipulating 
raster  images.  This  toolset  also  allows  researchers  to  convert  file  formats  among  over 
thirty  widely  used  graphics  formats  (e.g.,  from  HDF  to  PICT)  and  includes  extensive  C 
library  functionality  for  creating  custom  image-manipulation  applications.  Now  in 
its  second  release,  it  runs  on  Cray  Research,  DEC,  HP,  IBM,  SGI,  and  Sun  Microsys- 
tems platforms.  Binaries  and  sample  source  code  are  available  in  the  public  domain 
by  accessing  SDSC's  anonymous  ftp  area  (ftp.sdsc.edu). 

Scientist's  Workbench 

The  Scientist's  Workbench  is  an  X  and  Motif-based  software  package  developed  at  the 
CTC.  The  main  functions  of  the  Scientist's  Workbench  are  to  bring  together  the  tools 
and  software  required  by  scientific  researchers  in  a  distributed  computing  environ- 
ment, to  provide  a  graphical  interface  to  access  those  tools,  and  to  provide  the  soft- 
ware necessary  to  allow  researchers  to  easily  build  their  own  graphical  interfaces. 
This  tool  has  been  used  at  most  of  the  CTC's  Smart  Nodes  (affiliates)  by  users  and  as 
part  of  teaching  environments  for  high  performance  computing,  as  well  as  at  the 
other  centers  and  by  companies  and  national  labs  developing  "custom"  program- 
ming interfaces  for  their  communities. 
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Accon^plishments  in  Education  and  Outreach 

Education 

Familiarity  with  the  tools  of  computation  and  visualization  is  quickly  becoming  a 
sine  qua  non  for  both  researchers  and  the  public.  The  spread  of  access  to  these  tools, 
like  access  to  the  telephone  and  television  before  them,  is  a  democratizing  force  in 
itself:  the  world  of  the  shut-in  is  opened  up,  the  disadvantages  of  distance  are  mini- 
mized, the  exchange  of  techniques  and  knowledge  is  enhanced.  Education,  training, 
and  outreach  are  thus  fundamental  to  the  programs  of  the  MetaCenter.  Each  mem- 
ber of  the  centers  program  has  developed  educational  programs  targeted  to  a  variety 
of  constituencies:  university  researchers,  graduate  students,  undergraduates,  educa- 
tors at  all  levels,  and  K-12  students. 

Researchers  and  Students 

One-  or  two-day  workshops  are  offered  by  centers  program  staff  to  researchers  on  site 
and  at  associated  institutions,  covering  introductions  to  the  computational  envi- 
ronments, scientific  visualization,  and  the  optimization  and  parallelization  of  scien- 
tific code.  In  addition,  special  workshops  have  been  offered  throughout  the  centers 
program  on  the  use  and  extension  of  computational  and  visualization  techniques 
specific  to  various  disciplines  (from  biochemistry  to  lattice  gauge  theory).  On  the 
campuses  of  centers  program  institutions  and  on  other  campuses,  centers  program 
scientists  and  engineers  are  active  teachers,  either  through  regular  academic  ap- 
pointments or  as  adjuncts,  lecturers,  seminar  leaders,  or  teachers  in  extension  divi- 
sions. 

Graduate  students  often  receive  fellowships  or  similar  appointments  at  centers  pro- 
gram institutions,  as  their  contributions  may  benefit  a  large  academic  research 
community  or  the  computational  community  generally.  As  an  example,  Ph.D.  stu- 
dent Kimberly  Claffy  (Computer  Science,  UCSD)  recently  completed  her  dissertation 
on  a  flow-based  measure  of  Internet  traffic  that  she  developed  as  a  Junior  Fellow  at 
SDSC.  Dr.  Claffy's  technique  is  the  first  to  permit  traffic  characterization  on  the  basis 
of  a  temporally  and  spatially  flexible  unit,  and  it  is  thus  an  enabling  technology  for 
further  advanced  network  research  at  SDSC  and  elsewhere. 

The  centers  program  has  contributed  to  the  research  projects  of  hundreds  of  gradu- 
ate students  through  stipends,  access  to  resources,  and  relations  with  centers  pro- 
gram researchers.  Each  Center  fosters  collaborative  research  by  muitidisciplinary, 
multi-institutional  teams  of  computer  scientists,  research  scientists,  and  engineers; 
postdoctoral  research  associates;  and  graduate  students  from  the  national  and  inter- 
national community.  These  teams  forge  new  approaches  to  previously  insoluble  re- 
search problems,  develop  community  codes,  and  host  workshops  and  seminars  to 
transfer  technology. 

Supercomputer  Centers  Educational  Activity  Support  Summary 


Educational  Activities 
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FY92 

FY93 
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Institutes 

7 

4 

5 

Attendees 

128 

131 

121 

Other  K-12  Events 

15 

8 

17 

Attendees 

715 

1,370 

1,985 

Research  Institutes 

13 

11 

6 

Attendees 

262 

377 

390 

Training  Courses/ Workshops                                                                 | 

On  Site  Events 

18 

102 

134 

Attendees 

1,414 

1,773 

1,929 

Off  Site  Events 

17 

23 

17 

Attendees 

295 

622 

104 

Seminars/Colloquia 

Events 

138 

114 

132 

Attendees 

2,251 

2,788 

3,085 

Academic  Course  Accounts 

64 

63 

79 

Monthly  Newsletter  Circu- 
lation 

234,986 

247,692 

165,176 

Visitors 

13,506 

16,380 

16392 

For  undergraduates,  the  Research  Experiences  for  Undergraduates  programs,  funded 
by  NSF,  bring  in  undergraduates  to  work  for  a  summer  or  a  school  semester  or  quar- 
ter on  specific  projects  devised  by  centers  program  researchers  and/or  faculty  advi- 
sors. The  projects  are  significant  in  their  scope  of  computational  science  and  in 
many  instances  have  resulted  in  presentations  at  meetings  and  publications.  A  spe- 
cial project  is  CTC's  Supercomputing  Programs  for  Undergraduate  Research  (SPUR), 
in  which  students  apply  to  work  on  one  of  a  selection  of  projects  developed  by  Cor- 
nell faculty  in  collaboration  with  CTC.  One  REU  student  at  SDSC  went  on  to  win 
the  top  prize  in  the  Westinghouse  Science  Talent  Search  in  1991.  Another  devel- 
oped a  program  to  teach  the  use  of  the  Braillewriter  to  blind  students,  which  was 
presented  to  the  Commission  on  Equal  Opportunity  in  Science  and  Engineering  at 
NSF  (this  student,  herself  blind,  is  now  a  successful  computer  scientist  in  Silicon 
Valley).  Undergraduate  assistantships  and  internships  are  also  available  in  the  cen- 
ters program.  Undergraduate  student  programmers  have  worked  on  many  research 
problems  including  numerical  weather  prediction,  the  visualization  of  numerical 
spacetimes,  and  social  network  analysis.  They  have  developed  numerous  applica- 
tions and  utilities  to  improve  the  computational  environment  for  MetaCenter  re- 
searchers. Students  have  also  worked  on  library,  visualization,  and  educational  pro- 
jects. The  REU  programs  have  been  ongoing  in  various  forms  for  more  than  five 
years. 

Outreach  to  Educators 

One  particularly  effective  approach  to  educating  the  next  computational  generation 
is  the  training  of  teachers,  and  many  centers  program  efforts  have  been  devoted  to 
teacher  training  and  curriculum  development. 
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Common  Knowledge:  Pittsburgh  is  a  national  pilot  program  developed  by  PSC,  the 
University  of  Pittsburgh,  and  the  Pittsburgh  Public  Schools  to  institutionalize  educa- 
tional technologies  within  the  Pittsburgh  Public  School  District  by  having  PSC  im- 
plement the  network  infrastructure  and  develop  specific  curriculum-based  network 
and  computer  applications.  PSC's  High  School  Initiative  (1992-1994)  involves  stu- 
dent/teacher teams  using  PSC  facilities  to  develop  computational  tools  for  inclusion 
in  their  schools'  science  or  mathematics  curriculum,  with  an  emphasis  on  integrat- 
ing high-performance  computing  into  the  curriculum  and  thus  bridging  the  gap  be- 
tween textbook  instruction  and  real  world  applications  of  scienre. 

SuperQuest  is  a  program  involving  centers  program  sites  that  brings  teams  of  teach- 
ers and  students  from  selected  high  schools  to  summer  institutes  to  develop  compu- 
tational and  visualization  projects  that  they  work  on  throughout  the  following  year. 
In  addition  to  educational  workshop  programs  associated  with  SuperQuest,  NCSA 
has  developed  five  interactive  simulation  programs  now  being  tested  in  classrooms 
across  the  country  and  around  the  world.  These  include  GalaxSee,  an  N-body  simu- 
lator of  galaxy  formation  and  interaction;  the  Fractal  Microscope,  which  enables  the 
exploration  of  self-similar  patterns;  SimSurface  and  SimElevator,  simulated  anneal- 
ing programs;  and  LaplaceSeein',  an  electrostatic  potential  solver.  Students  can 
change  initial  conditions  and  watch  the  simulation  evolve  as  the  parameter  space  is 
explored. 

SDSC's  computational  sciences  curriculum  coordinator,  Kris  Stewart  (who  is  a  pro- 
fessor of  mathematics  at  San  Diego  State  University)  has  conducted  summer  work- 
shops, funded  by  NSF  and  Cray  Research,  with  faculty  from  primarily  undergradu- 
ate institutions  to  develop  ways  of  incorporating  high-performance  computing  into 
the  curriculum.  Stewart  uses  the  workshop  materials  in  her  own  SDSU  classes,  Su- 
percomputing  for  the  Sciences  and  an  Introduction  to  Computational  Analysis. 
SDSC  is  now  halfway  through  a  three-year,  NSF-funded  Supercomputer  Teacher 
Enhancement  Program  targeted  to  high-school  teachers  whose  classes  contain  un- 
derrepresented  minorities. 

Dr.  Bruce  Land,  of  the  CTC,  has  developed  an  undergraduate  course  in  scientific  vi- 
sualization and  computer  graphics,  using  the  data  flow  block  diagram  capabilities  of 
IBM's  Data  Explorer  software.  This  curriculum,  lab  exercises  and  the  resulting  stu- 
dent projects  have  been  shared  with  the  larger  educational  community  through  the 
CTC's  Education  and  Training  home  page.  Additionally,  Prof.  Steve  Vavasis  has  de- 
veloped an  interdisciplinary  course  in  scientific  computing  using  high  performance 
computing  for  graduate  and  undergraduate  students. 

Over  the  past  year,  the  CTC  had  established  the  Data  Explorer  repository,  a  full  set  of 
tutorials  for  parallel  computing  on  diverse  platforms,  a  complete  set  of  lecture  notes 
for  use  by  educators  as  well  as  researchers,  and  a  gateway  to  materials  on  the  net- 
work for  secondary  school  science  and  mathematics  education. 

The  educational  outreach  programs  of  the  centers  program  enable  students  to  expe- 
rience the  advantages  of  connectivity  and  training  in  all  aspects  of  modern  compu- 
tational practice.  The  challenge  to  effectively  deliver  centers  program  resources  to  all 
classrooms  is  being  met  mainly  by  the  distance-defeating  and  multiplicative  effects 

-22- 


143 


High  Performance  Computing  Infrastructure  and  Accomplishments 

of  high-performance  computation  itself.  The  dissemination  of  trairung  and  curricu- 
lum  materials  over   the  National   Information   Infrastructure   is  a  major    way  in 
which  the  successful  pilot  programs  can  be  turned  into  a  new  class  of  educational  re- 
sources. 
Outreach 

Because  advances  in  high-performance  computing  and  communications  (HPCC)  are 
driven  by  the  needs  of  the  practitioners  with  the  most  advanced  problems,  the  cen- 
ters program's  scientific  mission  includes  the  construction  of  an  extensive  web  of  re- 
lationships with  research  and  development  efforts  in  American  industry  and  com- 
merce. Collectively,  the  centers  program's  outreach  programs  represent  a  long 
record  of  sustained  collaboration  among  scientists,  HPCC  developers,  and  industrial 
researchers.  Another  aspect  of  outreach  is  the  effort  to  find  and  serve  local  and  re- 
gional needs  of  government,  schools,  and  communities.  Some  aspects  of  these  activ- 
ities are  discussed  below. 

Application  of  Scientific  Computation  and  Visualization  to  Industrial  Produc- 
tion 

Half  of  the  partnerships  between  the  individual  centers  and  industry  are  collabora- 
tions with  major  industrial  firms.  These  include  American  Cyanamid,  Amoco,  Al- 
coa, AT&T,  Caterpillar,  Corning,  Dow  Chemical,  Eastman  Kodak,  Eli  Lilly,  FMC, 
Gencorp,  General  Dynamics,  Hughes  Aircraft,  IBM,  JP  Morgan,  Martin-Marietta, 
McDormell-Douglas,  Merck  Research  Labs,  Motorola,  Parke-Davis,  Philips 
Petroleum,  Schlumberger,  USX,  and  Xerox. 

In  their  original  form,  the  partnerships  represented  the  first  introduction  of  large- 
scale  computation  and  visualization  into  the  store  of  resources  possessed  by  even 
the  largest  of  these  Fortune  500  comparues.  While  the  companies  are  for  the  most 
part  fully  computerized  now,  the  majority  of  these  partnerships  continue  today  be- 
cause centers  program  expertise  has  been  essential  to  the  introduction  of  new  ways 
of  employing  the  resources  of  supercomputing:  the  algorithms,  visualization  rou- 
tines, and  engineering  codes  are  being  combined  in  ways  that  result  in  such  ad- 
vances as  high-end  rapid  prototyping  of  new  products.  As  a  result,  for  example,  Eli 
Lilly  maintains  its  partnership  although  the  company  has  purchased  its  own  super- 
computer— the  useful  interactions  with  centers  program  scientists,  consultants,  and 
visualizers  continue.  In  many  of  these  arrangements,  the  industrial  partner's  re- 
searchers are  frequent  visitors  to  the  NSF  centers,  and  centers  program  researchers 
also  visit  the  partner's  installations. 

Thus,  while  it  is  extremely  important  to  Alcoa  that  it  was  able  to  produce  an  opti- 
mum aluminum  can,  to  Gencorp  that  it  was  able  to  design  a  better  injection  mold- 
ing process,  to  McDonnell-Douglas  that  it  could  perform  rapid  airfoil  analyses,  to 
American  Cyanamid  that  it  could  reformulate  soil  enhancers,  the  sum  of  these 
long-term  relationships  is  important  in  another  dimension  as  well.  TheCenter  out- 
reach efforts  are  helping  to  revitalize  American  industry,  making  it  more  competi- 
tive in  an  increasingly  competitive  world  market. 
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Impact  on  Vendors  of  High  Performance  Computing  Equipment 

The  centers  program  has  had  a  major  impact  on  the  vendors  of  major  high  perfor- 
mance computing  equipment.  All  Centers  have  taken  early  prototypes  of  machines, 
have  provided  national  access  to  the  largest  scale  version  of  such  machines,  and 
provided  critical  feedback  to  the  vendors.  Several  have  entered  into  strategic  devel- 
opment efforts  v^ith  the  vendors.  For  example,  PSC  is  a  formal  partner  with  Cray 
Research  in  the  development  of  applications  for  its  massively  parallel  T3D.  Cray 
has  also  internalized  some  of  PSC's  file  system  developments.  CTC  played  an  inte- 
gral role  in  IBM's  re-entry  into  the  High  Performance  Computing  arena,  as  the  first 
customer  for  its  IBM  ES/3090  vector  supercomputers  and  as  a  partner  in  the  design 
and  development  of  its  parallel  FORTRAN  products.  CTC,  through  its  director 
Malvin  H.  Kalos,  was  a  key  influence  in  IBM's  decision  in  1991  to  build  the  IBM  SP 
systems  and  ensured  that  IBM  adopted  a  strategy  that  was  scalable  ultimately  up  to 
the  teraflops  and  down  to  the  desktop. 

SDSC  has  established  a  close  collaboration  with  the  Supercomputer  Systems  Divi- 
sion (SSD)  of  Intel  Corporation  to  develop  systems  software  to  support  multi-user 
systems,  to  serve  as  a  test  site  for  new  operating  system  releases,  and  to  improve  the 
stability  of  the  Paragon  system.  SDSC  staff  have  developed  MACS,  the  Multi-user 
Accounting  and  Control  System,  which  Intel  offers  as  part  of  the  Paragon's  standard 
operating  system  software.  This  system  includes  a  dynamic  job  mix  scheduling  algo- 
rithm, a  port  of  the  Network  Queuing  System  batch  job  submission  software,  and 
CPU  quota  and  accounting  systems  to  control  resources  used  by  separate  projects. 

SDSC  has  also  collaborated  with  Cray  Research  to  develop  support  for  multiple-user 
systems  on  Cray  systems.  SDSC  staff  have  developed  a  resource  management  system 
that  controls  access  to  various  resources  on  the  system  (CPU,  memory,  and  disk)  and 
a  dynamic  job  mix  scheduler  (DJMS)  to  dynamically  adjust  the  workload  for  optimal 
performance.  SDSC  has  recently  run  a  T3D  emulator  on  the  Cray  C90  and  is  provid- 
ing feedback  on  its  performance.  SDSC  now  plans  to  install  and  evaluate  Cray's  new 
FDDI  card,  a  fiber-optic  high-speed  network  interface. 

Digital  Equipment  Corporation  recently  awarded  UCLA  and  SDSC  an  external  re- 
search grant  to  acquire  nine  Alpha  3000  model  400  workstations.  The  duster,  which 
has  a  peak  speed  of  1.2  Gflops  and  is  connected  at  100  Mbps  via  a  Gigaswitch,  will  be 
used  primarily  for  climate  studies  led  by  Dr.  Roberto  Mechoso  of  UCLA.  It  will  also 
be  available  for  scientific  use  and  performance  testing  by  the  SDSC  user  community. 
SDSC  staff  are  collaborating  with  DEC  to  port  the  global  climate  model  to  the  Alpha 
cluster  using  DEC's  High  Performance  Fortran  compiler. 

DEC  is  a  major  supporter  of  Project  Sequoia  2000,  a  collaboration  of  scientists,  com- 
puter and  information  experts,  government  agencies,  and  industrial  sponsors  to  de- 
velop an  information-management  system  for  studying  global  climate  change.  The 
Sequoia  visualization  group,  centered  at  SDSC,  has  been  developing  a  system  that 
will  build  on  the  strengths  of  existing  hardware  and  software  to  support  next-gener- 
ation visualizations.  Recently,  the  group  collaborated  with  Kubota-Padfic  Computer 
Corporation  to  combine  Kubota's  Denali  system  with  DEC  Alpha  machines  for  ad- 
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vanced  3D  color  graphics  capabilities.  Such  collaborations  have  benefited  DEC,  Kub- 
ota-Pacific,  and  the  Sequoia  2000  project. 

The  National  Storage  Laboratory  (NSL),  a  consortium  that  is  developing  next-gener- 
ation high-speed  storage  devices,  has  selected  the  UniTree  system  as  the  production 
archival  storage  system  for  all  centers  program  sites.  UniTree,  which  was  originally 
developed  at  the  Lawrence  Livermore  National  Laboratory,  was  commercialized  l^ 
DISCOS,  a  spin-off  of  SDSC  and  a  former  division  of  SDSC's  parent  company.  Gen- 
eral Atomics.  DISCOS  was,  in  turn,  sold  to  Open  Vision,  which  continues  to  market 
the  product. 

SDSC  staff  are  collaborating  with  NSL,  IBM,  and  Open  Vision  to  implement  the 
NSL's  base  version  of  the  UniTree  archival  storage  system  on  the  center's  IBM 
RS/6000  model  980  workstation.  They  are  adding  a  new,  more  robust  name  server 
developed  by  Lawrence  Livermore  National  Laboratory  for  Open  Vision,  transaction 
journaling  (which  allows  reconstructing  the  database  in  case  of  catastrophic  failure), 
and  adding  and  enhancing  system  administrator  tools. 

The  integration  of  PSC's  Multi-resident  AFS  into  the  NSL  UniTree  environment  is 
planned  to  provide  user-friendly  access  for  the  centers  program  members  into  SD- 
SC'S NSL  archival  storage  system.  The  future  integration  of  HIPPI-attached  periph- 
erals, including  high-speed,  high-density  tape  and  high-speed  RAID  disk  arrays,  us- 
ing third  party  data  transfer  is  planned  as  a  way  to  substantially  increase  both 
archival  system  storage  capacity  and  data  transfer  speed. 

A  prototype  HPSS  parallel  I/O  archival  storage  system  is  also  planned  for  evaluation 
as  the  follow-on  to  the  NSL  UruTree  system.  This  system  will  support  striping 
across  multiple  high-speed  peripherals  to  even  further  increase  the  speed  of  file 
transfers. 

Stimulation  of  New,  Computationally  Dependent  Ventures 

About  a  fourth  of  the  industrial  partnerships  are  with  smaller  and  newer  firms, 
many  of  them  leaders  in  biotechnology.  Some  are  firms  designing  new  pharmaceu- 
ticals (e.g.,  Agouron  Pharmaceuticals,  Genentech),  others  develop  and  market  the 
software  packages  required  for  these  enterprises.  Biosym  Technologies,  for  example, 
is  working  with  both  CTC  and  SDSC  to  develop  parallel  versions  of  its  popular  Dis- 
cover and  Insight  packages. 

Outreach  efforts  of  the  centers  program  have  resulted  in  actual  spinoff  ventures  as 
well.  The  commercialization  of  the  software  developed  at  individual  centers  is  be- 
ing undertaken  by  a  number  of  companies.  For  example,  NCSA  Telnet  has  been 
commercialized  by  Intercon,  and  Spyglass  will  release  a  package  containing  up- 
graded versions  of  image  tools  and  Mosaic.  Some  20  companies  have  now  licensed 
NCSA  Mosaic.  CERFnet,  a  California  wide-area  network  for  Internet  access  has 
pioneered  in  supplying  access  to  library  and  other  large  databases;  and 
DISCOS/UniTree,  a  mass  storage  system,  is  in  use  at  more  than  twenty  major 
computer  sites.  A  new  molecular  modeling  system,  called  Sculpt,  developed  at 
SDSC,  is  being  commercialized  by  a  new  company.  Interactive  Simulations.  Sculpt 
enables    drag-and-drop    molecular     modeling     in     real     time     while     preserving 
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minimum-energy  constraints;  its  output  was  featured  on  the  cover  of  Science  last 
May. 

Development  of  Nationally  Valuable  Reservoirs  of  Skill 

About  a  fourth  of  the  partnerships  between  the  centers  and  industry  are  collabora- 
tions with  manufacturers  of  high-performance  computational,  networking, 
telecommunications,  and  visualization  equipment.  Of  particular  interest  here  are 
the  several  partnerships  funded  by  the  NSF  and  ARPA  through  CNRI  to  construct 
and  test  the  "gigabit  testbeds,"  prototypes  of  the  connectivity  that  will  be  required  for 
the  future  International  Information  Infrastructiire.  Both  academic  and  industrial 
research  groups  are  developing  the  codes  to  test  the  connections,  even  as  manufac- 
turers develop  the  connections  themselves,  and  the  experts  assembled  in  the  centers 
program  supply  links  in  the  form  of  specifications  and  software. 

Community  Service 

Local  and  regional  outreach  efforts  range  from  the  tours  given  at  all  centers  program 
installations  through  the  hosting  of  visits  by  national,  regional,  and  local  officials 
and  commissions,  to  the  kinds  of  full-scale  partnerships  mentioned  above.  The 
NCSA  relationship  with  the  Champaign  County  Chamber  of  Commerce  has  re- 
sulted in  the  formation  of  a  nonprofit  public  network,  CCnet,  which  is  already  bene- 
fiting the  Chamber  itself  as  well  as  local  schools.  Plans  are  in  the  works  with  Time- 
Warner  to  start  pilot  tests  of  the  use  of  public-access  cable  for  a  data  highway. 

SDSC  is  working  with  the  City  of  San  Diego  on  plans  to  connect  all  units  of  dty  gov- 
ernment, including  a  high-technology  resource  center  to  be  developed  with  De- 
partment of  Commerce  funding  that  will  connect  local  industry  (with  a  lot  of  de- 
fense reconversion  efforts)  to  business  and  computational  resources,  including 
SDSC  itself. 

PSC  is  exploring  extension  of  the  technology  it  developed  for  its  Common  Knowl- 
edge:Pittsburgh  K-12  project  to  embrace  major  units  of  city  government. 

Outreach  is  also  represented  by  the  publications  programs  of  the  centers  program, 
the  production  of  scientific  videos  and /or  multimedia  CD-ROMs,  and  a  collabora- 
tive program  for  maintaining  a  lively  and  informative  presence  on  World-Wide 
Web  servers,  which  will  make  information  on  the  programs  easily  accessible  over 
the  Nn. 
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Important  Science  and  Engineering  Accomplishments 

Summaries  of  computationally  interesting  problems  in  the  NSF  Centers  Program  by 
the  national  science  and  engineering  communities: 

QUANTUM  PHYSICS  AND  MATERIALS 

The  great  disparity  between  nuclear,  atomic,  or  molecular  scales  and  macroscopic 
material  scales,  implies  that  vast  computing  resources  are  needed  to  attempt  to  pre- 
dict the  characteristics  of  bulk  matter  from  fundamental  laws  of  physics.  Therefore, 
it  is  not  surprising  that  since  the  begirming  of  the  NSF  Centers  program  this  area  of 
science  has  brought  us  some  of  the  largest  users  of  supercomputers.  Materials  scien- 
tists have  often  been  among  the  first  group  of  researchers  to  try  out  new  architec- 
tures that  promise  even  higher  computational  speeds. 

Below  are  outlined  some  outstanding  examples  of  studying  properties  of  bulk  mat- 
ter from  extreme  conditions,  such  as  occur  in  nuclear  collisions,  the  early  universe, 
or  in  the  core  of  Jupiter;  new  materials  such  as  nanotubes  and  high  temperature  su- 
perconductors; and  more  practical  materials  used  today  such  as  magnetic  material 
and  glass. 

Phase  Transition  in  QCD 

The  MIMD  Lattice  Calculations  Collaboration  (MILC)  is  attacking  the  Grand  Chal- 
lenge problem  of  "the  origins  of  mass."  Their  objective  is  to  use  the  theory  of  the 
forces  governing  what  are  called  the  "strong  interactions"  of  elementary  particles 
(quarks  and  gluons)  to  calculate  the  observed  masses  and  interactions  of  the  particles 
that  are  made  out  of  them:  the  hadrons,  which  include  the  familiar  proton  and  neu- 
tron. The  theory  is  called  quantum  chromodynamics  (QCD),  and  its  numerical  in- 
carnation is  called  "lattice  gauge  theory,"  because  the  quarks  and  gluons  are  repre- 
sented on  a  four-dimensional  space-time  lattice.  They  have  published  numerous 
studies  of  the  mass  spectrum  of  the  hadrons;  the  transition  between  ordinary  matter 
and  the  quark-gluon  plasma,  which  is  important  in  the  study  of  the  conditions  of 
the  early  uruverse;  and  the  decays  of  hadrons  via  weak  interactions.  A  number  of 
investigators,  coordinated  by  Robert  Sugar  (Dept.  of  Physics,  UCSB),  are  engaged  in 
this  project  including:  Claude  Bernard  (Washington  Uruv.),  Thomas  A.  DeGrand 
(Univ.  of  Colorado),  Carleton  DeTar  (Univ.  of  Utah),  Steven  Gottlieb  and  Alexander 
Krasnitz  (Indiana  Univ.),  Douglas  Toussaint  (Univ.  of  Arizona),  Julius  Kuti  (UCSD). 
The  consortium  has  used  large  allocations  of  time  on  a  wide  range  of  MetaCenter 
computational  facilities  including:  Intel  Paragon  (SDSC),  TMC  CM-5  (NCSA),  clus- 
tered IBM  RS/6000S  under  PVM  (CTC,  NCSA),  Cray  Research  C-90  (SDSC  and  PSC) 

Phase  Transitions  of  Solid  Hydrogen 

Calculations  by  Natalie,  Martin  and  Ceperley  (Dept.  of  Physics  UIUC,  NCSA),  carried 
out  on  the  CRAY  Y-MP  at  NCSA  have  established  the  series  of  crystalline  phase 
transitions  of  hydrogen  as  it  is  compressed  to  several  million  atmospheres  of  pres- 
sure, such  as  found  in  the  interior  of  the  giant  planets.  Since  Wigner  and  Hunting- 
ton in  1935  pointed  out  that  a  transformation  from  a  molecular  to  atomic  state  is  in- 
evitable at  high  pressure,  there  have  been  extensive  speculations  on  when  and  how 
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this  transformation  would  take  place.  The  recent  development  of  the  diamond 
anvil  technique  have  allowed  experiments  to  be  performed  at  pressure  slightly 
lower  than  the  atomic  transition.  Those  experiments  confirmed  the  existence  of  an 
molecular  orientation  transition  which  had  been  earlier  computationally  predicted 
by  Ceperley  at  l.SMbar.  Extensive  and  highly  accurate  quantum  Monte  Carlo  calcu- 
lations on  a  variety  of  crystal  structures  now  predict  that  the  metallic  transition  will 
take  place  from  a  distorted  molecular  hexagonal  structirre  into  an  atomic  diamond 
lattice.  For  this  and  other  pioneering  science,  David  Ceperley  was  awarded  the  fifth 
Eugene  Feenberg  Memorial  Silver  Medal  in  1994.  David's  PhD  advisor  and  the  third 
Feenberg  awardee  is  Mai  Kalos,  Director  of  the  CTC,  himself  a  major  user  of  several 
MetaCenter  supercomputers. 

Prediction  of  new  Nanomaterials 

Marvin  L.  Cohen  (NAS)  and  Steven  G.  Louie  (Dept.  of  Physics,  UC  Berkeley)  have 
used  MetaCenter  computational  resources  (SDSC,  PSC,  NCSA  Cray  Research  Y-MP 
and  C90)  to  make  numerous  advances  in  computational  materials  science.  Most  re- 
cently, they  have  used  both  first-principles  and  tight-binding  codes  to  examine  the 
properties  of  carbon  nanotubes  and  nanotubes  composed  of  boron,  carbon,  and  ni- 
trogen. Carbon  nanotubes-essentially  rolled  microsheets  of  graphite-are  well 
known,  thanks  to  the  work  of  Sumio  lijima  and  colleagues  at  NEC.  They  have  di- 
ameters on  the  order  of  1-20  nm  and  are  producible  in  the  same  carbon  arc  chambers 
used  to  produce  fullerenes  (also  called  "buckyballs"-  assemblages  of  60  or  more  car- 
bon atoms  in  cage-like  structures).  They  have  interesting  capillarity  and  electronic 
properties.  Cohen,  Louie  and  their  colleagues  have  predicted  the  structure  and 
properties  of  nanotubes  made  of  boron  nitride,  which  appear  to  be  more  stable  and 
controllable  in  terms  of  their  electronic  properties.  They  have  also  predicted  nan- 
otubes of  BC_2N,  a  boron-carbon-nitrogen  compound,  whose  electroruc  properties 
are  even  more  interesting:  they  should  behave  like  nanoscale  induction  coils.  Most 
exciting,  the  structures  that  were  predicted  computationally  are  now  being  produced 
experimentally  in  the  lab  of  Alex  Zettl  at  Berkeley,  where  their  electronic  properties 
can  be  confirmed. 

Theory  of  High  Temperature  Superconductors 

The  Nobel  Prize  in  Physics  in  1987  was  for  the  discovery  of  a  new  class  of  high  tem- 
perature superconductors.  Thousands  of  research  papers  have  been  written  about 
these  unique  materials,  but  the  battle  is  still  raging  over  the  fundamental  mecha- 
nism that  causes  the  superconducting  transition  at  Tc  ~  90K  for  the  cuprate  oxides 
such  as  YBA2CU3O7.  David  Pines  (NAS  and  first  Feenberg  Medalist)  and  Philippe 
Monthoux  (Department  of  Physics,  UIUC)  used  the  NCSA  Cray  Research  Y-MP  to 
carry  out  a  strong  coupling  (Eliashberg)  calculation  of  the  normal  state  properties 
and  Tc  for  the  model  experiment-based  magnetic  interaction  between  quasiparticles. 
They  found  that  when  the  full  structure  of  the  quasiparticle  interaction  is  taken  into 
account,  a  superconducting  transition  into  a  d-wave  planar  pairing  state  occurs  at  Tc 
~  90K  for  comparatively  modest  values  of  the  coupling  constant.  Although  still  an 
area  of  active  research,  this  computation  lends  credibility  to  the  model  that  it  is  the 
coupling  of  planar  quasiparticles  to  the  experimentally  measured  planar  electronic 

-28- 


149 


High  Performance  Computing  Infrastructure  and  Accomplishments 

spin  fluctuation  excitations  which  determines  the  normal  state  properties  (which 
they  show  acts  like  a  nearly  antiferromagnetic  Fermi  liquid)  and  makes  possible 
high  temperature  superconductivity. 

Magnetic  Materials 

James  Sethna  (Laboratory  of  Atomic  and  Solid  State  Physics,  Cornell  Univ.)  uses 
CTC  parallel  supercomputers  to  study  the  dynamics  of  disorder-driven  first-order 
phase  transformations,  including  3-D  numerical  simulations  of  hysteresis  loops.  He 
is  developing  scalable  parallel  algorithms  for  systems  of  size  N  =  20003.  Sethna's 
work  enables  the  prediction  of  phase  transitions  with  critical  fluctuations,  the  simu- 
lation of  orders-of-magnitude  larger  systems  to  explore  critical  phenomena,  and  de- 
tailed computational  studies  in  materials  science  as  applied  to  magnetic  storage  me- 
dia, metallurgical  phase  transformations,  and  gases  adsorbed  on  surfaces. 

Understanding  Glass 

For  ab  initio  dynamical  calculations  to  be  useful  for  real  materials  in  an  industrial 
setting,  they  must  be  able  to  deal  with  ensembles  of  thousands  of  atoms  for  dynami- 
cal effects  modeled  over  microseconds.  Significant  algorithmic  developments  made 
jointly  at  Corning,  Inc.  and  the  CTC,  coupled  with  the  much  increased  capability  of 
the  CTC's  IBM  SP2  system,  allow  this  threshold  to  be  crossed  for  the  first  time.  Post- 
doctoral Fellow  Stefan  Goedecker,  hired  jointly  by  Corning,  Inc.  and  CTC,  has  devel- 
oped new  extremely  fast  ways  of  doing  tight-binding  which  he  can  parameterize 
with  the  ab  initio  codes.  This  is  the  only  approach  currently  known  that  will  handle 
thousands  of  atoms  for  millions  of  time  steps,  bringing  the  researchers  close  to  ob- 
serving many  of  the  mysteries  involved  in  glass  chemistry  which  have  been  not 
well  understood  for  over  2000  years. 

Biology  AND  Medicine 

Living  creatures  exhibit  some  of  the  greatest  complexity  found  in  nature.  Therefore, 
supercomputers  have  made  possible  unprecedented  opportunities  to  explore  these 
complexities  based  on  the  fundamental  advances  made  in  biological  research  of  the 
last  fifty  years.  These  activities  include:  inverting  the  data  from  x-ray  crystallography 
experiments  to  obtain  the  molecular  structure  of  macromolecules;  learning  how  to 
use  artificial  intelligence  to  fold  polypeptide  chains,  determined  from  genetic  se- 
quencing, into  the  three-dimensional  proteins;  and  determining  the  function  of 
proteins  by  studying  their  dynamic  properties,  as  well  as  how  they  interact  with  each 
other  or  with  the  DNA  backbone  from  whence  they  were  created. 

These  insights  are  beginning  to  make  significant  impacts  on  medicine  and  plant  and 
animal  biology.  New  fields  of  computational  science,  such  as  molecular  neuro- 
sciences,  are  being  enabled  by  academic  access  to  MetaCenter  computing  and  visual- 
ization resources  and  staff.  Corporations  are  using  supercomputers  and  advanced 
visualization  techniques  in  collaboration  with  the  NSF  MetaCenter  to  create  new 
drugs  to  fight  human  diseases  such  as  asthma.  New  ir\sights  into  economically 
valuable  bioproducts  are  being  gained,  for  instance,  by  combining  molecular  and 
medical  imaging  techniques  to  create  "virtual  spiders"  which  can  be  digitally  dis- 
sected to  understand  the  production  of  silk.    Finally,  high  performance  computers 
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are  just  becoming  powerful  enough  that  some  dedicated  researchers  are  able  to  pro- 
gram mathematical  models  of  realistic  organ  dynamics,  such  as  the  human  heart. 

Crystallography 

Herbert  Hauptman  (Medical  Foundation  of  Buffalo,  Inc.)  won  a  Nobel  Prize  in  1985 
for  development  of  the  "direct  method"  of  protein  structure  determination  from  X- 
ray  crystallographic  data.  In  a  collaboration  with  Russ  Miller  (State  University  of 
New  York  at  Buffalo),  these  researchers  have  developed  a  numerical  approach  that 
extends  the  "direct  method"  of  determining  molecular  structure  from  X-ray  crystal- 
lographic data  to  larger  molecules,  beyond  its  present  limit  of  about  100  atoms.  The 
algorithm  they  have  developed,  called  "Shake-and-Bake,"  runs  on  a  number  of 
computing  platforms  (PSC  CM-2  and  Cray  T3D,  NCSA  and  PSC  CM-5)  and  has 
proven  itself  effective  in  more  than  20  cases  at  accurately  determining  the  structure 
of  proteins  that  have  taken  as  long  as  10  years  by  existing  methods,  reducing  the 
time  to  a  matter  of  hours. 

Folding  Proteins  using  Artificial  Intelligence 

One  of  the  most  pressing  problems  in  molecular  biology  is  how  to  determine  the 
folding  and  3-D  structure  of  a  protein,  given  its  sequence.  Peter  Wolynes  (NAS),  Zan 
Schulten,  and  coworkers  (Dept.  of  Chemistry,  UIUC)  have  developed  a  novel  ap- 
proach to  this  classic  problem  using  elements  from  the  theory  of  spin  glasses,  asso- 
ciative memory  models,  and  neural  networks.  Spin  glass  theory  provides  a  frame- 
work for  understanding  the  cooperative  nature  of  the  folding  transition  and  the 
qualitative  nature  of  the  phase  diagram  describing  the  thermodynamics  of  proteins. 
Wolynes  et  al.  developed  simulation  codes,  based  on  associative  memory  Hamilto- 
nians,  and  characterized  their  phase  diagrams  semi-quantitatively.  These  Hamilto- 
nians  are  based  on  an  energy  function  which  correlates  the  sequence  of  the  protein 
to  be  folded  with  those  of  proteins  whose  structure  is  known.  They  were  introduced 
several  years  ago  by  Wolynes  and  coworkers  as  polymer  analogues  of  the  Hopfield 
neural  nets.  Their  work,  carried  out  on  NCSA's  Cray-2  and  CRAY  Y-MP,  shows  that 
even  primitive  associative  memory  Hamiltonians  can  recognize  protein  structures 
from  sequences  that  are  only  moderately  related  to  those  already  existing  in  the 
database.  These  procedures  are  somewhat  similar  in  effectiveness  to  the  rule-based 
homology  modeling. 

Protein  Kinase  solution 

The  Computational  Center  for  Macromolecular  Structure  (CCMS),  founded  in  1990, 
is  an  NSF-fundcd  joint  project  of  UCSD,  SDSC,  and  The  Scripps  Research  Institute, 
with  collaborators  from  all  over  the  country.  The  center  made  headlines  in  1991 
when  a  group  led  by  Susan  Taylor  (Dept.  of  Chemistry,  UCSB),  one  of  the  principal 
investigators  of  CCMS  published  the  three-dimensional  structure  of  the  catalytic 
unit  of  cyclic-AMP-dependent  protein  kinase,  or  cAPK.  This  was  the  first  kinase 
structure  to  be  solved.  The  solution  was  achieved  by  a  combination  of  computa- 
tional methods,  including  refinement  on  the  SDSC  CRAY  Y-MP  using  the  program 
XPLOR,  developed  by  Axel  Bruenger  of  Yale  University.  Most  important  to  the  solu- 
tion was  the  ability  of  the  scientists  to  study  stereo  visualizations  of  the  structure  on 
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a  large  screen  at  SDSC,  so  scientists  from  every  discipline  within  the  group  could 
contribute  their  insight  to  a  collective  determination  of  the  structure.  Kinases  play 
important  messenger  roles  in  cell  metabolism,  and  hundreds  of  such  compounds 
have  been  identified  and  sequenced.  Because  sequences  are  homologous  in  long 
stretches,  the  solution  for  cAPK  is  proving  extremely  valuable  as  a  template  for 
modeling  and  deriving  the  structure  of  other  kinases.  Taylor  and  her  group  have 
collaborated  with  several  other  groups  since  in  modeling  proposed  solutions  for 
other  kinases,  including  those  known  to  have  carcinogenic  properties  or  to  be  in- 
volved in  other  disease  processes.  In  all  of  these  studies,  computation  and  visualiza- 
tion have  played  an  important  role.  Solutions  for  various  kinases  can  lead  to  the  de- 
sign of  inhibitors  to  prevent  the  enzymes  from  acting  to  produce  diseases.  The  work 
won  the  Forefronts  of  Large-Scale  Computation  Award  presented  at  Supercomput- 
ing  '93. 

Molecular  Neuroscience-Serotonin 

A  number  of  cardiovascular  and  psychiatric  diseases  are  currently  treated  with  drugs 
that  act  on  the  neurotransmitter  serotonin  and  its  receptors.  The  cellular  receptor 
for  serotonin  is  a  gatekeeper  molecule  that  recognizes  and  binds  the  serotonin  and 
then  transmits  the  signal  to  the  cell  by  binding  to  a  special  class  of  transducers:  the 
G-proteins.  Using  the  CTC's  ES/9000,  Dr.  Harel  Weinstein,  chairman  of  the  Depart- 
ment of  Physiology  and  Biophysics  at  Mount  Sinai  Medical  Center,  made  a  break- 
through in  modeling  the  serotonin  receptor.  His  breakthrough  came  from  modeling 
the  structural  changes  that  occur  in  the  serotonin  receptor  when  it  binds  to  a  ligand 
and  to  the  G-protein,  causing  it  to  carry  out  its  function.  This  research  shows  how 
G-proteins  can  be  switched  on  by  structural  changes  in  specific  regions  of  the  recep- 
tor molecule  and  is  expected  to  lead  to  the  development  of  more  effective  drugs, 
specific  ligands  aimed  at  the  regions  where  the  structural  change  takes  place.  Wein- 
stein believes  that  his  work  may  be  applied  more  broadly  to  other  receptor 
molecules,  including  all  neurotransmitters,  that  communicate  with  cells  via  G-pro- 
teins. If  Weinstein  can  demonstrate  a  common  mechanism  of  response  in  these  re- 
ceptors, he  will  have  a  new  typ)e  of  molecular  approach  to  treating  a  vast  range  of 
diseases.  Weinstein  has  also  used  the  Cray  Research  C-90  and  Intel  Paragon  at  SDSC. 

Molecular  Neuroscience-Acetylcholinesterase 

A  collaboration  between  Michael  Gilson,  T.P.  Straatsma,  and  Andrew  McCammon 
(Dept.  of  Chemistry,  University  of  Houston),  Daniel  RipoU  (Research  Associate, 
CTC),  Carlos  Faerman  (Dept.  of  Molecular  and  Cell  Biology,  Cornell  Univ.),  Paul 
Axelsen  (Dept.  of  Pharmacology,  University  of  Pennsylvania  School  of  Medicine), 
and  Israel  Silman  and  Joel  Sussman  (Weizmann  Institute  of  Science,  Israel)  has 
used  molecular  dynamics  algorithms  to  investigate  the  rapid  activity  of  the  enzyme 
acetylcholinesterase  (AChE).  The  enzyme  breaks  down  the  neurotransmitter  acetyl- 
choline diffused  across  nerve  cell  synaptic  gaps.  Its  three-dimensional  crystal  struc- 
ture was  solved  by  Joel  Sussman  and  colleagues  at  the  Weizmann  Institute  in  Re- 
hovot,  Israel,  several  years  ago.  That  structure  showed  the  active  site  to  be  a  long, 
narrow  channel — too  narrow  to  deal  rapidly  with  the  job  of  dissociating  acetyl- 
choline into  choline  and  an  acetate  ion.  Yet  it  is  known  that  AChE  acts  very  rapidly. 
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no  doubt  because  of  aeons  of  evolutionary  pressure  to  optimize  the  response  of  the 
nervous  system  in  all  organisms.  Molecular  dynamics  calculations  performed  on 
the  MetaCenter's  Cray  Research  C-90  (SDSC  and  PSC),  TMC  CM-5  (NCSA),  and 
Kendall  Square  KSR-1  (CTC)  showed  that  there  was  also  a  "back  door"  to  the  active 
site,  that  might  open  to  facilitate  the  exit  of  the  acetate  ion  from  the  site.  A  study  of 
the  electromagnetic  fields  of  acetylcholinesterase  with  the  back  door  closed  and  then 
open  supplied  confirming  evidence.  Since  inhibitors  of  AChE  are  important  medi- 
cations for  myasthenia  gravis,  glaucoma,  and  Alzheimers's  disease,  this  new  insight 
may  lead  to  more  effective  pharmaceutical  agents  to  fight  these  diseases.  This  work 
was  the  cover  story  of  the  March  4,  1994  issue  of  Science  magazine. 

Kinking  DNA 

John  M.  Rosenberg  (University  of  Pittsburgh)  used  the  PSC  Cray  Research  C-90  vec- 
tor supercomputer  to  determine  how  a  protein  identifies  and  interacts  with  specific 
sites  of  DNA  —  a  fundamental  biological  process  called  "protein-DNA  recognition," 
which  is  related  to  many  disease  processes  and  is  also  a  vital  tool  in  the  biotechnol- 
ogy industry.  Rosenberg's  molecular  dynamics  simulations  have  refined  the  struc- 
ture of  an  important  protein,  Eco  Rl  endonuclease,  used  in  DNA  cloning,  and  they 
have  resulted  in  a  clear  understanding  of  a  "kink"  in  the  DNA  backbone  that  results 
when  Eco  Rl  endonuclease  binds  with  DNA.  Rosenberg  won  the  1991  Forefronts  of 
Large-Scale  Computation  award  for  this  research,  and  his  work  was  cited  in  the  1993 
Computerworld  Smithsonian  award  for  science  given  to  the  PSC. 

Antibody-Antigen  Docking 

A  collaboration  among  computer  scientists  Michael  Hoist  and  Faisal  Saied  (Dept.  of 
Computer  Science,  UIUC)  and  two  biologists  Richard  Kozack  and  Shankar  Subra- 
maniam  (Dept.  of  Physiology  and  Biophysics,  UIUC  and  Beckman  Institute/NCSA) 
has  been  able  to  solve  for  the  first  time  the  complete  nonlinear  Poisson-Boltzmann 
equation,  which  is  the  fundamental  equation  of  macromolecular  electrostatics.  A 
method  based  on  multigrid-inexact  Newton  algorithms  has  been  developed  and 
large  memory  applications  run  in  parallel  on  the  NCSA  Convex  C3  show  that  this 
has  profound  consequences  for  protein  structure,  enzyme  mechanisms  and  protein 
design.  Coupling  this  new  approach  with  a  Brownian  dynamics  method,  the  largest 
simulation  ever  of  an  encounter  between  two  proteins,  an  antibody  and  an  antigen, 
has  been  carried  out  using  the  NCSA  CM-5  and  SGI  Challenge.  This  simulation  for 
the  first  time  is  able  to  give  rate  constants  for  association  of  proteins  that  is  compa- 
rable to  experimental  measurements.  The  results  of  the  electrostatics  work  was  the 
cover  story  of  the  March  1994  issue  of  Proteins:  Structure,  Function,  and  Genetics. 

Tuning  Biomolecules  to  Fight  Asthma 

Over  the  last  20  years,  the  number  of  asthma  cases  has  almost  tripled  in  the  U.S. 
David  Herron,  senior  research  scientist  at  Eli  Lilly  and  Company,  is  searching  for 
new  drugs  that  will  inhibit  the  action  of  leukotrienes,  inflammatory  agents  released 
by  several  types  of  cells  in  the  lungs,  which  cause  the  lungs  to  stiffen  and  become  ir- 
ritated. Several  gigabytes  of  data  from  molecular  dynamics  of  three  key 
leukotrienes,  run  on  NCSA's  and  Lilly's  Cray-2  supercomputer  were  analyzed  in  a 
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lengthy  scientific  visualization  created  with  NCSA  staff.  Using  the  animations  as  a 
guide,  Lilly  asthma  researchers  synthesized  highly  active  antagonists  against  the 
leukotrienes.  Some  have  been  tested  in  asthma  sufferers  and  found  to  be  effective 
medically.  For  this  work,  Herron  was  a  co-recipient  of  the  first  NCSA  Industrial 
Grand  Challenge  Award. 

Virtual  Spider  and  Artificial  Silk 

Biophysicist  Lynn  Jelinkski,  director  of  Cornell  University's  Biotechnology  Center 
for  Advanced  Technology  (CAT)  is  combining  medical  imaging  techniques  with  the 
state-of-the-art  computer  visualization  resources  of  the  CTC  to  study  the  molecular 
structiire  of  the  strongest  silk  of  the  golden  orb  weaver  spider  and  its  transformation 
from  a  viscous  fluid  into  the  extremely  strong  crystalline  fiber  which  has  the  poten- 
tial to  replace  manmade  fibers,  such  as  nylon,  manufactured  from  petrochemicals. 
Jelinski  has  devised  a  way  to  create  a  3-D  "computer  spider"  by  compiling  stacks  of 
the  2-D  MRI  images  using  the  IBM  POWER  Visualization  System,  one  of  the  high- 
performance  computing  resources  of  the  CTC.  Each  image  contains  over  100,000 
pixels.  Hundreds  of  images  are  combined  to  construct  the  3-D  simulated  spider. 
Once  in  hand,  this  virtual  spider  can  be  dissected  by  computer  to  describe  the 
anatomy  of  the  glandular  system  and  to  provide  the  physical  processing  informa- 
tion Jelinski  seeks.  This  understanding,  coupled  with  molecular-level  studies  of  the 
amino  acids  that  make  up  the  web  silk  polymer,  may  aid  in  geneticaUy  engineering 
plants  to  produce  fibers  as  strong  as  those  produced  by  the  spider.  Jelinski's  work 
blazes  a  path  toward  the  development  of  a  new  class  textiles  with  superior  strength 
at  the  same  time  that  it  promises  fundamental  insight  into  the  mystery  of  the  spi- 
der's web. 

Heart  Modeling 

Charles  S.  Peskin  and  David  M.  McQueen  (Courant  Institute,  New  York  University) 
have  developed  over  the  last  decade  a  fully  functioning  three-dimensional  model  of 
the  heart,  its  valves  and  nearby  major  vessels.  This  computational  model  will  make 
it  possible  to  study  questions  about  normal  and  diseased  heart  function  that  are  dif- 
ficult or  impossible  to  address  through  animal  and  clinical  studies.  The  complexity 
of  the  heart  model  is  so  great  that  a  single  heartbeat  requires  a  150  hour  run  on  the 
PSC  Cray  Research  C90  and  could  not  have  been  run  without  the  very  large  memory 
of  the  C90.  This  research  won  the  1994  Computerworld  Smithsonian  award  for 
Breakthrough  Computational  Science.  Peskin  was  awarded  a  MacArthur  Prize  Fel- 
lowship in  1983. 

Engineering 

Man-made  devices  have  become  so  complex  that  researchers  in  both  academia  and 
industry  have  tvirned  to  supercomputers  in  order  to  be  able  to  analyze  and  modify 
accurate  models  in  ways  which  complement  the  traditional  experimental  methods. 
Such  easily  accessible  high  performance  computers  enable  academic  engineers  to 
study  the  brittleness  of  new  types  of  steel,  to  improve  bone  transplants,  or  to  reduce 
drag  of  flows  over  surfaces  using  riblets.  Industrial  partners  of  the  individual  super- 
computer centers  within  the  MetaCenter  are  using  computational   facilities  more 
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advanced  than  they  have  access  to  internally  to  improve  industrial  processes  such  as 
in  metal  forming.  Better  consumer  products  such  as  leakproof  diapers,  or  more  effi- 
cient airplanes  are  being  designed.  Even  State  agencies  are  able  to  use  the  MetaCen- 
ter  facilities  to  improve  traffic  safety  or  find  better  ways  to  use  recycled  materials. 
Some  70  corporations  have  taken  advantage  of  the  MetaCenter  industrial  programs 
to  improve  their  competitiveness. 

Ultra-high-strength  Steels 

Gregory  B.  Olson  and  Arthur  J.  Freeman  (Northwestern  University)  use  computer 
modeling  to  design  ultra-high  strength  steel  for  weight-critical  applications  such  as 
naval  aircraft  landing  gear,  high-performance  race  cars,  and  bearings  in  the  main 
engine  turbo  pumps  of  the  space  shuttle.  In  recent  supercomputer  modeling  on  the 
PSC  Cray  C90,  applying  quantum  mechanical  calculations  to  the  structure  of  steel, 
they  have  explained  the  molecular  mechanisms  that  give  rise  to  impurity-induced 
embrittlement  in  steel,  work  which  is  expected  to  lead  to  steel  that  will  not  shatter 
in  frigid  conditions.  This  work  was  reported  in  the  July  15,  1994  Science.  Freeman 
and  his  group  have  been  users  of  NSF  supercomputers  since  the  founding  of  the 
program  on  a  wide  range  of  problems  in  materials  sciences.  In  recognition  of  his  pi- 
oneering work  in  computational  materials  research.  Freeman  received  the  first  Ma- 
terials Research  Society  Medal  and  the  first  Award  in  Magnetism  from  the  lUPAP. 

Continuous  Casting  of  Steel 

Achilles  Vassilicos  (U.S.  Steel  Technical  Center)  models  the  flow  of  molten  steel  in  a 
continuous-casting  "tundish"  on  the  PSC  Cray  Research  C90,  resulting  in  improved 
process  control  over  the  quality  of  steel.  By  more  acciirately  predicting  the  precise 
metallurgical  composition  of  the  continuous-casting  output  "slabs,"  U.S.  Steel  re- 
duces waste  steel  and  the  amount  of  inventory  it  must  keep  on-hand,  resulting  in 
substantial  cost  savings. 

Beverage  can  design 

Three-dimensional  stress  modeling  of  aluminum  beverage  cans  on  the  PSC  C90  by 
Robert  E.  Dick  and  Andrew  B.  Trageser  (ALCOA  Laboratories)  has  greatly  reduced 
the  expense  of  developing  a  new  can  design  that  will  meet  customer  specifications 
for  strength  and  appearance.  By  relying  less  on  costly,  time-consuming  prototype 
testing,  ALCOA  engineers  estimate  a  cost  savings  of  $100,000  or  more  per  can  design. 
This  research  has  been  described  in  articles  in  Discover  (March  1991),  Business  Week 
(Oct.  8,  1990)  and  in  Science  Qune  23,  1989). 

Designing  a  Leakproof  Diaper 

Designing  effective  and  comfortable  disposable  infant  diapers  requires  greater  un- 
derstanding of  the  function  of  the  diaper  components,  such  as  the  cellulosic  fluff 
and  the  superabsorbent  polymer  particles-and  the  effect  of  variations  of  parameters 
related  to  these  components.  Dow  Chemical  Company,  has  done  extensive  experi- 
mental testing,  evaluation,  and  computer  modeling  that  has  contributed  to  a  faster 
developmental  process  and  shortening  the  time  for  new  product  introduction.  The 
innovative   Dow  design  was  evaluated  using  a  computer  model  run  on  NCSA's 
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CRAY-2  system.  Three  separate  time-dependent  processes  are  modeled.  The  first,  a 
fast  spreading  process,  involves  the  insult  on  the  pad  by  a  quantity  of  liquid  (urine) 
which  is  transported  through  the  pad  by  wicking.  During  this  process  the  fluff  pad 
collapses  as  it  becomes  wet.  The  second,  called  the  imbibition  process,  models  the 
swelling  of  the  superabsorbent  polymer  particles  and  their  uptake  of  liquid  from  the 
cellulose  fluff.  During  this  process,  the  fluff  expands  again.  The  slowest-and  final- 
process  tracks  the  redistribution  of  liquid  in  the  fluff  pad  as  the  saturation  of  the 
fluff  adjacent  to  the  superabsorbents  changes.  The  overall  model  was  compared  to  a 
magnetic  resonance  imaging  experiment,  which  provides  a  three-dimensional  im- 
age of  the  water  distribution  in  a  diaper,  and  was  shown  to  give  comparable  results 
to  the  final  steady-state  values.  Optimization  of  these  processes  is  leading  to  an  im- 
proved, quality  diaper. 

Bone  Transplant  Bioengineering 

Dean  Taylor  and  Donald  Bartel  (Dept.  of  Mechanical  and  Aerospace  Engineering, 
Cornell  University)  have  been  able  to  investigate  bone-implant  systems  across  a 
wide  range  of  design  parameters  by  using  high  performance  parallel  computing 
(including  the  IBM  SPl)  and  visualization  resources  at  CTC.  Their  long-term  re- 
search has  produced  models  of  the  stresses  placed  on  normal  bones  and  the  artificial 
components  of  a  hip  joint — these  models  have  led  to  custom-designed  prostheses 
and  reduced  prosthesis  replacement  surgery. 

Improving  Performance  with  Riblets 

George  Em  Karniadakis^  and  his  group  at  Brown  University  are  using  the  SDSC  In- 
tel Paragon  to  explain  drag  reduction  in  turbulent  flow  by  means  of  "riblets,"  paral- 
lel grooves  on  the  surface  of  an  object  moving  through  the  flow.  Such  grooves  are 
used  on  aircraft,  in  pipelines,  and  on  radng  cars  and  sleds,  to  improve  performance. 
The  group  also  models  flows  in  micro-electromechanical  systems  used  in  surgery 
and  other  complex  applications,  where  the  molecules  of  the  fluid  are  not  much 
smaller  than  the  channels  in  which  they  flow.  Both  projects  are  resulting  in  new 
ways  to  optimize  the  performance  of  broad  classes  of  machinery. 

Designing  Better  Aircraft 

Dino  Roman,  John  Vassberg,  and  Tom  Gruschus  of  McDonnell  Douglas  are  using 
SDSC's  Cray  C-90  for  a  computational  fluid  dynamics  simulation  of  an  aircraft  in 
flight  and  to  visualize  the  results  using  FAST  (Flow  Analysis  Software  Toolkit)  on  a 
Silicon  Graphics  IRIS  workstation.  Tracer  particles  are  released  into  the  flow  field  in 
front  of  the  aircraft  and  allowed  to  follow  the  streamlines  around  the  vehicle.  A  cut- 
ting plane  through  the  3D  volume  of  data  is  placed  to  intersect  the  aircraft  fuselage 
and  wings.  The  aerospace  industry  relies  on  computational  fluid  dynamics-the 
simulation  of  air  or  fluid  flow-to  design,  develop,  and  test  new  aeronautical  config- 
urations. This  process  enables  companies  to  test  new  models  quickly  to  select  candi- 


' Incidentally,  Prof.  Kamiadakis  is  the  current  chair  of  the  joint  NCSA/PSC  National  Peer  Review 
Board,  another  example  of  the  tight  links  between  members  of  the  MetaCenter  and  the  scientific 
community. 
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dates  for  wind-tunnel  testing.  Such  methods  used  in  aircraft  design  and  manufac- 
ture can  give  American  companies  a  technological  edge  in  the  global  market. 

Crash  Testing  Street  Signs 

California  State  Department  of  Transportation  (Caltrans)  engineer  Payam  Rowhani 
and  SDSC  engineer  Chuck  Charman  generated  a  computerized  model  of  a  crash-test 
vehicle,  called  a  "bogie,"  on  the  SDSC  Cray  Research  Y-MP  for  simulations  of  test 
crashes  with  sign  and  lighting  supports.  They  fine-tuned  the  bogie  front-end  design 
with  the  computer  model  to  minimize  the  number  of  validation  tests  necessary  at 
the  Federal  Outdoor  Impact  Laboratory,  operated  by  the  Federal  Highway  Adminis- 
tration. In  a  second  application,  Charman  is  working  with  Caltrans  engineers 
William  Nokes  and  Dario  Perdomo,  who  are  designing  pavement  using  structural 
modeling  techniques.  They  are  researching  the  use  of  new  and  recycled  materials 
and  these  materials'  response  to  different  axle  and  tire  configurations.  They  are  us- 
ing the  supercomputer  and  the  visualization  facilities  to  explore  new  truck  suspen- 
sion systems,  and  new  tires  and  heavier  loadings,  innovative  pavement  structures 
with  recycled  materials  and  rubber  and  polymer-modified  binders.  The  results  of 
this  research  are  expected  to  lead  to  significant  cost  savings  in  the  design,  construc- 
tion, maintenance,  and  rehabilitation  of  pavement  structures. 

Earth  Sciences  and  the  environment 

From  understanding  the  motions  of  the  Earth's  convective  mantle  to  daily  compu- 
tation of  air  pollution  levels  in  southern  California,  the  resources  of  the  NSF  Meta- 
Center  are  being  used  to  compute  and  visualize  the  complexity  of  the  natural  world 
around  us.  The  US  Army  is  working  with  academics  to  determine  how  they  can 
practice  tank  maneuvers  without  endangering  the  breeding  habits  of  the  sage 
grouse.  Pollution,  whether  underground  or  in  the  air,  is  a  difficult  coupling  of 
chemical  reactions  and  flow  dynamics  which  must  be  understood  in  detail  if  correc- 
tive measures  are  to  be  efficacious.  High  performance  computers  also  act  as  time 
machines,  allowing  for  faster-than-realtime  computation  of  severe  storms.  Finally, 
to  improve  global  weather  or  climate  forecasts,  supercomputers  allow  researchers  to 
zero  in  on  the  critical  coupling  physics  of  such  processes  as  mixing  at  the  air/ocean 
interface. 

Detoxification  of  Ground  Water 

Christine  Shoemaker  (Dept.  of  Civil  and  Environmental  Engineering,  Cornell  Uni- 
versity) has  been  a  pioneer  user  of  the  scalable  IBM  SP  machines  at  the  CTC  for  the 
development  of  numerically  efficient  supercomputer  algorithms  for  optimal  con- 
trol of  dynamical  systems  and  the  application  of  these  techniques  to  detoxification  of 
contaminated  groundwater.  Her  efforts  are  leading  to  methods  of  determining  the 
most  cost-effective  way  to  clean  up  the  groundwater  by  computing  time-varying 
rates  of  pumping.  Shoemaker's  group  has  also  developed  an  animation,  using  the 
visualization  resources  (both  hardware  and  personnel)  of  the  CTC,  that  represents 
the  effects  of  different  policies  and  natural  chemical  and  biological  processes  on 
groundwater  cleanup.  Such  animations  are  crucial  for  conveying  the  results  of  basic 
research  to  the  mixed  audience  involved  in  setting  environmental  policy. 


-36 


157 


High  Performance  Computing  Infrastructure  and  Accomplishments 
Sage  Grouse-Endangered  Species  and  the  US  Army 

Working  with  the  U.  S.  Army,  Bruce  Hannon  (Dept.  of  Geography,  UIUC)  and  Jim 
Westervelt,  of  the  U.  S.  Army  Construction  Engineering  Research  Laboratory,  have 
developed  an  ecological  model  for  the  sage  grouse,  an  endangered  species,  popula- 
tion on  an  army  training  base  in  Washington  State.  Using  the  Macintosh  software 
STELLA,  the  CM-5  and  the  GRASS  geographical  information  systems,  these  re- 
searchers were  able  to  optimize  the  scheduling  of  trairung  exercises  to  maximize 
grouse  reproduction  and  longevity.  For  each  geographic  cell,  a  very  large  STELLA 
model  was  constructed,  representing  the  grouse  at  various  life  stages,  different  kinds 
of  plants  and  predators,  soil  type  and  moisture,  weather-all  the  many  physical  vari- 
ables-and  also  introduce  the  necessary  human  activities  like  tank  and  troop  maneu- 
vers on  the  army  range.  Each  cell,  of  which  there  are  over  a  hundred  thousand  i  n 
the  GIS  covering  the  army  base,  could  contain  100  to  200  variables.  This  work 
demonstrated  the  efficacy  of  coupling  GIS  datasets  to  ecological  models  and  running 
them  in  a  client-server  fashion  between  a  Macintosh  and  the  CM-5. 

Storm  modeling/forecasting 

Robert  Wilhelmson  (Dept.  of  Atmospheric  Sciences,  UTUC  and  NCSA)  and  his  col- 
leagues have  been  able  to  simulate  the  development  of  tornadoes  embedded  within 
larger  storms  called  supercells  (producing  the  largest  tornadoes)  and  along  low  level 
convergence  boundaries  (e.g.  along  a  thunderstorm  cold  air  boundary)  using  both 
traditional  vector  supercomputers  (NCSA  and  PSC  Cray  Research  Y-MP  and  C90  and 
NCSA  TMC  CM-2  and  CM-5).  Study  of  these  results  is  leading  to  a  better  under- 
standing of  when  tornadoes  will  develop  and  to  more  accurate  tornado  warnings. 
The  visualization  of  the  internal  dynamics  of  a  severe  thunderstorm,  created  by  the 
NCSA  visualization  team  in  1989,  is  perhaps  the  most  widely  viewed  visualization 
of  a  supercomputer  simulation  ever  made.  It  had  a  major  worldwide  impact  on  the 
adoption  of  scientific  visualization  as  a  working  tool  of  computational  science. 

Kelvin  Droegemeier,  a  former  student  of  Wilhelmson's,  and  his  colleagues  associ- 
ated with  the  Center  for  Analysis  and  Prediction  of  Storms  (CAPS),  an  NSF  S  &  T 
Center,  have  used  the  NCSA  and  PSC  Cray  Research  supercomputers  to  develop  the 
Advanced  Regional  Prediction  System,  a  computational  model  for  forecasting  se- 
vere storms.  As  of  Spring  1994,  this  model  has  been  used,  with  data  augmented  by 
the  single-Doppler  radar  network  now  being  deployed  by  NOAA,  in  daily  weather 
reporting  on  an  experimental  basis.  Because  of  their  use  of  parallel  supercomputers 
they  have  shown  that  regional  storm  forecasts  based  on  very  high  resolution  mod- 
els are  possible  with  the  advent  of  teraflop  computing  capabilities  in  the  next  few 
years.  The  long-term  objective  is  to  improve  the  prediction  of  hazardous  weather  on 
scales  ranging  from  a  few  kilometers  (an  individual  storm)  and  tens  of  minutes  to 
hundreds  of  kilometers  (a  squall  line  or  other  mesoscale  system)  and  several  hours. 

Los  Angeles  Smog 

Gregory  J.  McRae  (Massachusetts  Institute  of  Technology)  and  Armistead  Russell 
(Carnegie  Mellon  University)  have  developed  the  most  comprehensive  model  of 
smog  formation  available.  Their  modeling  of  smog  in  Los  Angeles  on  the  PSC  C90 
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showed  that,  contrary  to  EPA  policy  at  the  time,  it  is  necessary  to  control  nitrogen 
oxide  emissions  as  well  as  hydrocarbons  to  control  smog.  This  work  formed  the  sci- 
entific underpinning  for  the  Air  Quality  Management  Plan  adopted  in  1988-89  for 
the  Los  Angeles  air  basin,  the  most  stringent  such  plan  in  the  United  States.  Their 
modeling  also  showed  that  alternative  vehicle  fuels,  methanol  in  particular,  repre- 
sent a  worthwhile  strategy  for  improving  urban  air  quality,  which  influenced  inclu- 
sion of  this  policy  in  the  1990  revisions  to  the  Federal  Clean  Air  Act.  This  work  is  be- 
ing extended  using  the  combination  of  the  PSC  Cray  Research  C90  and  T3D  as  an 
NSF  Grand  Challenge.  The  first  Forefronts  of  Large-Scale  Computation  award,  given 
in  1989,  recognized  McRae  for  this  work. 

These  pioneering  computations  are  leading  to  practical  tools  for  states  to  predict  air 
pollution  levels.  The  Modeling  and  Meteorology  Branch  of  the  California  Air  Re- 
sources Board  joined  the  SDSC  Industrial  Partners  program  in  1991.  They  are  run- 
ning, on  the  SDSC  Cray  Research  C90,  the  Urban  Airshed  model  which  estimates 
hourly  pollutant  concentrations.  They  results  are  used  to  estimate  maximum  pollu- 
tant concentrations  or  population  exposure  statistics  for  different  emissions  con- 
trols. 

Upper  Ocean  Mixing 

Sidney  Leibovich  (Dept.  of  Mechaiucal  and  Aerospace  Engineering,  Cornell  Univer- 
sity) has  developed  a  mathematical  model  on  the  CTC  IBM  ES/9000  and  SPl  that 
uses  a  series  of  equations  to  simulate  what  happens  when  wind  blows,  waves  form, 
and  Langmuir  circulations  begin  to  mix  the  upper  ocean.  This  mixing  has  a  marked 
influence  on  the  density  of  the  water;  the  ocean's  density  structure  alters  its  current 
patterns,  and  the  combined  phenomena  influence  the  exchange  of  heat  between  the 
ocean  and  the  atmosphere.  All  of  this  has  environmental  and  ecological  conse- 
quences, and  finding  ways  to  predict  this  chain  of  events  is  of  interest  to  ecologists, 
oil  companies,  meteorologists,  climatologists,  undersea  communications  experts, 
and  government  policy  makers. 

Simulating  Climate  using  Distributed  Supercomputers 

C.  Roberto  Mechoso,  University  of  California,  Los  Angeles;  Larry  Bergman,  Jet 
Propulsion  Laboratory;  Carl  Scarbnick,  Gary  Hanyzewski,  and  Bilal  Chinoy,  SDSC; 
Paul  Messina,  California  Institute  of  Technology;  and  Robert  Malone  and  Rick 
Smith,  Los  Alamos  National  Laboratory  are  developing  a  coupled  general  circula- 
tion model  distributed  over  a  heterogeneous  network.  The  atmospheric  component, 
which  runs  on  JPL's  CRAY,  is  a  finite-difference  model  with  state-of-the-art  parame- 
terizations  of  convection,  planetary  boundary  layer  processes,  and  radiation.  It  pro- 
vides wind  stress  and  heat  flux  information  to  the  firute-difference  oceanic  compo- 
nent, which  runs  on  Caltech's  Delta  or  SDSC's  Paragon.  The  oceanic  component 
then  returns  the  sea  surface  temperature  to  the  atmospheric  component.  Results  of 
the  coupled  system  show  a  very  realistic  seasonal  cycle.  This  is  a  significant  step  in 
the  development  of  a  coupled  system  distributed  across  heterogeneous  computer 
environments,  a  main  goal  of  the  CASA  gigabit  testbed  project. 
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PLANETARY  SCIENCES,  ASTRONOMY,  AND  COSMOLOGY 

The  sciences  of  the  world  beyond  the  Earth  have  always  been  an  interaction  of  ob- 
servation of  unexpected  events  and  theory  creating  a  model  built  on  the  laws  of 
physics  and  chemistry  which  explain  the  observations.  Therefore,  it  is  proper  to  see 
high  performance  computing  and  communications  making  major  impacts  on  both 
observation  and  theory.  Indeed,  as  we  saw  in  the  recent  impact  of  Comet  Shoe- 
maker-Levy 9  with  Jupiter,  the  observatories  on  earth  and  in  space  have  become  in- 
timately linked  with  each  other  and  with  the  theoretical  simulations.  Supercom- 
puters are  becoming  integrated  into  observational  facilities,  like  the  Grand  Chal- 
lenge BIMA  millimeter  observatory,  and  with  observational  programs  like  the  ones 
which  have  led  to  discovery  of  new  millisecond  pulsars  or  the  first  extrasolar  system 
planets,  themselves  orbiting  a  pulsar. 

The  ability  of  numerical  methods  to  solve  even  the  most  complex  of  fundamental 
physical  laws,  such  as  Einstein's  equations  of  General  Relativity,  is  leading  to  a  very 
rapid  understanding  of  the  dynamics  of  strong  field  events,  such  as  the  collision  of 
black  holes.  In  perhaps  the  grandest  scale  challenge  possible,  the  universe  itself  is  a 
subject  of  investigation  by  a  several  Grand  Challenge  teams  using  resources  of  the 
MetaCenter  to  discover  how  the  large  scale  structures  in  the  uruverse  evolved  from 
nearly  perfect  homogeneity  at  the  time  of  the  formation  of  the  microwave  back- 
ground. 

Comet  Collision  with  Jupiter 

Mordecai-Mark  Mac  Low  (Dept.  of  Astronomy  &  Astrophysics,  Univ.  of  Chicago  and 
Dept.  of  Astronomy,  UIUC)  and  Kevin  Zahnle  (Space  Science  Division,  NASA 
Ames  Research  Center)  simulated  the  impact  of  Comet  Shoemaker-Levy  9  on 
Jupiter  using  the  astrophysical  gas  dynamics  code  ZEUS-3D,  developed  at  the  NCSA, 
running  on  the  PSC  Cray  Research  C90.  Their  simulations,  made  months  before  the 
actual  impact,  were  widely  used  by  planetary  astronomers  to  prepare  for  their  ob- 
serving runs.  Their  results  showed  that  the  comet  would  penetrate  only  shallowly 
into  the  planet's  atmosphere  before  exploding  and  that  the  explosion  would  break 
out  of  the  atmosphere.  Their  animations  were  available  using  the  global  Mo- 
saic/WWW infrastructure  and  received  worldwide  coverage  in  both  the  popular 
and  scientific  press. 

Discovery  of  First  Extrasolar  System  Planet 

A  statistical  analysis  by  Alexander  Wolszczan  (Dept.  of  Astronomy  and  Astro- 
physics, Perm  State  Univ.)  of  data  received  from  the  radio  telescope  at  Arecibo  Ob- 
servatory in  Puerto  Rico  led  him  to  theorize  the  existence  of  planets  orbiting  around 
a  pulsar.  Searching  millions  of  bits  of  information  using  the  supercomputer  facili- 
ties at  the  CTC,  he  detected  an  unusual  complexity  in  the  pattern  of  the  pulses'  ar- 
rival time.  The  behavior  suggested  that  the  pulsar's  motion  is  affected  by  the  pres- 
ence of  other  orbiting  objects.  Applying  a  perturbation  model  proposed  by  Frederic 
Rasio  at  Princeton  Urviversity,  along  with  Philip  Nicholson,  Stuart  Shapiro,  and 
Saul  Teukolsky  at  Cornell  Univ.  to  a  new  set  of  observations,  Wolszczan  found  that 
it  predicted  the  effect  that  two  orbiting  planets  would  have  upon  each  other  as  they 
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moved  through  space.  The  model  fits,  and  Wolszczan  can  be  credited  with  identify- 
ing the  first  planetary  system  beyond  our  own. 

The  April  22  1994  issue  of  Science  featured  this  work,  with  front  page  coverage  of 
news  also  in  the  New  York  Times  that  week  as  well  as  several  other  major 
metropolitan  newspapers,  television,  and  popular  magazines.  Wolszczan  continues 
to  explore  this  new  world  using  parallel  capabilities  of  the  IBM  SP  machines  at  the 
CTC. 

Building  the  BIMA  Radio  Telescope 

This  HPCC  Grand  Challenge  project  at  NCSA  is  implementing  a  prototype  of  the 
next  generation  of  astronomical  telescope  systems  -  remotely  located  telescopes  con- 
nected by  high-speed  networks  to  very  high  performance,  scalable  architecture  com- 
puters and  on-line  data  archives,  which  are  accessed  by  astronomers  over  Gbit/sec 
networks.  Data  taken  by  the  BIMA  telescope  in  northern  California  are  now  trans- 
ferred in  real  time  to  NCSA  (using  a  combination  of  high-speed  leased  line  and 
NSFnet),  entered  into  a  database  program,  and  archived  on  the  NCSA  mass  storage 
system.  Astronomers  locate  data  in  the  archive  and  load  data  onto  NCSA  or  remote 
computers  for  processing  using  the  NCSA  Mosaic  software.  Also,  a  digital  library  of 
fully  processed  radio  images,  which  will  include  an  all-sky  radio  survey  created  by 
the  NRAO's  VLA  with  a  size  of  1  terabyte,  is  being  developed.  One  of  the  major  ac- 
tivities that  BIMA  and  NCSA  staff  are  involved  in  is  the  creation  of  a  new  standard 
software  package  AIPS++,  written  in  C++  for  extensibility  and  maintainability,  to 
replace  the  older  standard  packages  AIPS,  which  was  written  in  FORTRAN.  The 
AIPS++  software  is  being  developed  by  an  international  collaboration  including 
BIMA  member  institutions,  the  National  Radio  Astronomy  Observatory,  and  scien- 
tists from  Australia,  Canada,  England,  the  Netherlands  and  India.  With  AIPS++,  an 
astronomer  can  learn  the  system  once  and  then  process  images  on  nearly  any  com- 
puter using  data  from  almost  any  radio  telescope.  NCSA  will  implement  computa- 
tionally intensive  routines  on  the  SGI  Power  Challenge.  Finally,  discussions  are  un- 
derway to  have  the  BEMA  telescope  array  be  part  of  the  development  of  a  very  large 
national  millimeter  array  radio  telescope,  in  cooperation  with  the  Caltech  and  the 
National  Radio  Astronomy  Observatory. 

Pulsar  Searching  and  Discovery 

Joseph  Taylor  (Dept.  of  Astrophysical  Sciences,  Princeton  Univ.  and  1993  Nobel 
prize  winner  in  Physics)  and  his  colleagues  used  the  PSC  C90  to  aid  in  the  search  for 
pulsars  by  analyzing  radio-frequency  data  gathered  at  the  Arecibo  Observatory.  Since 
using  supercomputing  to  sift  through  the  enormous  amount  of  data  obtained,  this 
approach  has  resulted  in  the  discovery  of  20  previously  unknown  pulsars,  including 
five  millisecond  pulsars. 

Accretion  Disks  Around  Black  Holes 

Most  galaxies  seem  to  harbor  massive  black  holes  (between  one  million  and  ten  bil- 
lion solar  masses)  at  their  centers.  When  gas  accretes  onto  these  holes  enormous 
energies  can  be  released,  turning  the  most  luminous  of  these  objects  into  quasars. 
John  Hawley  (Dept.  of  Astronomy,  Univ.  of  Virginia)  has  been  using  supercomput- 
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ers  for  over  a  decade  to  investigate  the  accretion  disks  which  form  around  black 
holes.  Using  the  Cray  Research  Y-MP  and  C90  at  NCSA  and  PSC,  Hawley  developed 
a  new  magnetohydrodynamic  (MHD)  code  which  enabled  him  and  Steven  Balbus  to 
explore  the  nature  of  a  powerful  MHD  instability  in  accretion  disks.  They  discov- 
ered that  this  almost  universal  instability  is  the  underlying  physical  mechanism 
which  gives  rise  to  the  effective  turbulent  viscosity  in  the  disk,  which  previously 
theorists  had  only  modeled  parametrically.  For  this  discovery,  Hawley  received  the 
1993  Helen  B.  Warner  Prize  from  the  American  Astronomical  Society. 

Black  Hole  Collision  Dynamics 

The  two-body  problem  in  General  Relativity  is  represented  by  the  motion  of  two 
black  holes.  In  contrast  to  the  two-body  problem  in  Newtonian  gravity  which  was 
solved  in  dosed  form  over  300  years  ago,  the  orbiting  motion  of  two  black  holes  is 
an  outstanding  research  frontier,  recently  being  funded  by  the  NSF  as  a  Grand  Chal- 
lenge. Two  research  teams  in  this  Grand  Challenge,  Ed  Seidel,  NCSA  Director  Larry 
Smarr,  and  coworkers  (Dept.  of  Physics  and  NCSA,  UIUC)  and  Stu  Shapiro  and  Saul 
Teukolsky  (Dept.  of  Astronomy  and  Physics,  Cornell  University)  have  used  a  variety 
of  supercomputers  at  NCSA,  PSC,  and  CTC  to  compute,  for  the  first  time,  the  evolu- 
tion of  the  event  horizon  representing  the  head-on  collision  of  two  black  holes.  The 
spacetime  diagram  of  the  collision  shows  a  picture  like  the  predicted  "pair  of  pants", 
analyzed  qualitatively  by  Stephen  Hawking  and  others  in  the  early  1970s,  but  the  su- 
percomputer computation  gives  for  the  first  time  quantitative  details  about  the  area 
increase  and  the  caustics  formed  as  the  null  geodesies,  which  generate  the  horizon, 
intersect.  Seidel  and  coworkers  show  that  in  general  such  violent  deformations  of 
black  holes  cause  the  horizon  to  oscillate  with  the  quasinormal  modes  previously 
studied  only  in  linear  perturbation  theory.  The  Grand  Challenge  team  is  now  mov- 
ing on  to  the  full  spiraling-in  situation,  which  is  expected  to  be  one  of  the  first 
events  observed  by  the  LIGO  gravitational  wave  observatory  late  in  this  decade. 

Largest  cosmological  simulation 

This  simulation,  carried  out  by  Michael  Norman  and  Greg  Bryan  (NCSA  and  Dept. 
of  Astronomy),  on  the  TMC  CM-5  is  the  first  which  is  sufficiently  accurate  to  allow 
direct  comparison  with  x-ray  telescopic  observations,  thereby  allowing  a  critical  test 
of  this  popular  cosmological  model.  A  block  of  the  universe  half  a  billion  light  years 
on  a  side  was  evolved  from  the  era  of  the  formation  of  the  microwave  background 
(with  tiny  inhomogeneties  as  quantified  by  the  COBE  satellite)  until  the  present.  A 
single  evolution  required  30  hours  on  the  512-node  CM-5  and  over  14GB  of  memory 
to  hold  the  model.  The  results  were  announced  at  the  May  1994  meeting  of  the 
American  Astronomical  Society  and  covered  in  the  popular  press  by  CBS-TV  and 
the  Washington  Post.  The  computation  is  one  of  a  number  now  being  made  by 
members  of  the  Grand  Challenge  Cosmology  Consortium  using  resources  of  the 
MetaCenter.  Jerry  Ostriker  (NAS),  Chair  of  the  Department  of  Astrophysical  Sci- 
ences at  Princeton  University  is  The  largest  simulation  of  the  formation  of  x-ray 
clusters  in  a  universe  dominated  by  a  mixture  of  cold  and  hot  dark  matter  was  car- 
ried out  using  NCSA's  512  node  CM-5.  PI  of  the  GC^. 


■41 


162 


High  Performance  Computing  Infrastructure  and  Accomplishments 
EVOLUTION  OF  THE  METACENTER  CONCEPT 

As  the  decade  of  the  1990s  opened,  it  became  clear  that  we  were  witnessing  funda- 
mental changes  in  the  way  researchers  use  high-performance  computing  and  com- 
munications systems,  driven  by  technological  advances  in  computers  and  network- 
ing. Anticipating  these  changes,  the  four  NSF  supercomputing  centers  formed  a  col- 
laboration based  on  the  concept  of  a  national  MetaCenter  for  computational  science 
and  engineering:  a  collective  of  intellectual  and  physical  resources  unlimited  by  ge- 
ographical or  technological  constraint. 

The  primary  objective  was  to  realize  ways  in  which  researchers  could  be  best  served 
and  their  problems  directed  to  that  architecture  or  combination  of  architectures  best 
suited  to  their  solution,  on  a  nationwide  basis.  Thus  the  MetaCenter,  begun  in  1992, 
was  conceived  as  a  "garden  of  architectures"  that  could  demonstrate  ways  of  provid- 
ing nationwide  resources  to  support  scientific  and  teclinological  advances  and  sup- 
ply a  framework  within  which  the  NSF  centers  program  could  move  into  the  next 
era  of  high-performance  computing.  In  1994,  NCAR's  Scientific  Computing  Divi- 
sion joined  the  MetaCenter.  In  addition,  the  NSF  established  a  program  of  MetaCen- 
ter Regional  Affiliates  (MRA)  under  which  other  institutions  could  pursue  projects 
of  interest  in  collaboration  with  MetaCenter  institutions;  MetaCenter  sites  were  in- 
volved with  several  MRA  proposals. 

MetaCenter  projects  involve  the  use  of  the  resources  and  expertise  available  at  more 
than  one  MetaCenter  site.  Often  these  projects  are  grand  challenges  or  national  chal- 
lenges—many have  been  highlighted  elsewhere  in  this  report. 

In  addition  to  direct  support  of  researchers,  the  MetaCenter  sites  have  joined  to 
work  on  specific  projects  of  benefit  to  the  scientific  community.  Projects  begun  by 
the  MetaCenter  during  the  first  two  years  include: 

•  National  File  System:  Separate  cells  have  been  established  at  each  site  based 
on  the  Andrew  File  System  (AFS);  a  national  cell  that  will  serve  as  the  basis  for  a 
production  system  is  being  implemented.  With  such  a  system  in  place,  users  will 
be  able  to  move  transparently  among  the  MetaCenter  computational  systems. 

•  National  Archival  Storage  System:  Work  has  begun  on  a  proposed  archival 
storage  system  based  on  manufacturer-supported  and  MetaCenter-enhanced  prod- 
ucts, including  versions  of  UniTree  and  the  Data  Migration  Facility  from  Cray  Re- 
search. With  such  a  system  in  place,  files  on  any  center  machine  can  be  migrated 
transparently  among  other  center  machines,  the  national  file  system,  and  the  na- 
tional archival  storage  system. 

•  Video  Collaboratory:  Using  video  teleconferencing  equipment,  the  re- 
searchers and  staff  of  MetaCenter  institutions  meet  regularly,  often  with  participa- 
tion from  the  NSF.  The  equipment  has  been  used  to  broadcast  joint  training 
workshops  like  the  MetaCenter  Computational  Science  Institutes  in  Parallel 
Computing  conducted  at  SDSC  and  NCSA.  Plans  are  to  install  additional  hard- 
ware to  strengthen  this  system. 
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•  Visualization  Collaboration:  MetaCenter  and  NSF  Science  and  Technology 
Centers  researchers  participated  in  a  DASC -funded  MetaCenter  Visualization 
Workshop  held  at  SDSC  in  1993,  and  experiments  have  begun  in  passing  visual- 
izations between  the  centers  in  real  time  to  determine  bottlenecks  and  potential 
problems. 

•  Online  Information:  An  online  information  system  using  the  gopher  proto- 
cols has  been  implemented;  it  is  now  being  converted  to  be  accessible  via  World- 
Wide  Web  browsers.  Plans  are  to  broaden  the  information  available  and  incorpo- 
rate new  technologies.  A  joint  Science  Highlights  Repository  will  soon  include 
WWW  links  to  MetaCenter-supported  research  projects  worldwide. 

•  Networking:  Plans  were  developed  to  upgrade  the  high-speed,  wide-area 
network  connecting  the  centers  into  a  very  high  speed  backbone. 

New  projects  for  the  MetaCenter  include  exploring  virtual  reality  for  use  in  the  sci- 
entific discovery  process,  use  of  the  basis  developed  in  the  Oriline  Information  pro- 
ject to  develop  a  national  training  program  to  promote  understanding  and  expertise 
in  the  use  of  parallel  computing,  and  a  joint  evaluation  of  the  scalable  parallel 
computers  acquired  by  the  NSF  centers.  A  project  has  also  been  initiated  to  establish 
a  security  research  and  response  team  to  monitor  and  coordinate  responses  to  inci- 
dents of  security  violations. 

Recognition  Accorded  NSF  supercomputer  Users  and  Projects 

The  previous  sections  on  Accomplishments  of  the  NSF  Supercomputer  Program 
cite  several  major  ways  in  which  the  research  of  the  centers  has  been  recognized. 
Specific  notable  citations  and  users  are  summarized  here,  with  the  complete  descrip- 
tion available  in  the  appropriate  sections.  In  addition,  scientific  visualizations  pro- 
duced at  the  centers  have  provided  popular  recognition  as  covers  of  scientific  maga- 
2dnes  and  technical  programs,  as  evidenced  by  numerous  covers  of  Science,  and 
NSF's  own  publication  Mosaic. 

Phase  Transitions  of  Solid  Hydrogen  ...  David  Ceperley  was  awarded  the  fifth  Eu- 
gene Feenberg  Memorial  Silver  Medal  in  1994  [in  part  for  this  work  performed  at 
NSF  centers].  David's  PhD  advisor  and  the  third  Feenberg  awardee  is  Mai  Kalos,  Di- 
rector of  the  Cornell  Center. 

Protein  Kinase  solution  ...  (Susan  Taylor  et  al)  won  the  Forefronts  of  Large-Scale 
Computation  Award  presented  at  Supercomputing  '93  for  this  work... 

Kinking  DNA  ...  John  Rosenberg  won  the  1991  Forefronts  of  Large-Scale  Computa- 
tion award  for  this  research,  and  his  work  was  cited  in  the  1993  Computerworld 
Smithsonian  award  for  science  which  was  given  to  the  Pittsburgh  Supercomputer 
Center  (PSC). 

Heart  Modeling  ...  This  research  by  Charles  Peskin  won  the  1994  Computerworld 
Smithsonian  award  for  Breakthrough  Computational  Science.  Peskin  was  previ- 
ously awarded  a  MacArthur  Prize  Fellowship  in  1983,  for  earlier  studies  in  heart 
modeling. 
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Ultra-high-strength  Steels  Gregory  B.  Olson  and  Arthur  J.  Freeman  (Northwestern 
University)  use  computer  modeling  to  design  ultra-high  strength  steel  for  weight- 
critical  applications  ...  This  work  was  reported  in  the  July  15,  1994  Science.  Freeman 
and  his  group  have  been  users  of  NSF  supercomputers  since  the  founding  of  the 
program  on  a  wide  range  of  problems  in  materials  sciences.  Arthur  Freeman  re- 
ceived the  first  Materials  Research  Society  Medal  and  the  first  Award  in  Magnetism 
from  the  International  Union  of  Pure  and  Applied  Physics. 

Los  Angeles  Smog  ...  The  first  Forefronts  of  Large-Scale  Computation  award,  given 
in  1989,  recognized  Gregory  McRae  for  this  work. 

Theory  of  High  Temperature  Superconductors  ...The  Nobel  Prize  in  Physics  in  1987 
was  for  the  discovery  of  a  new  class  of  high  temperature  superconductors.  Thou- 
sands of  research  papers  have  been  written  about  these  unique  materials...  David 
Pines  (NAS  and  first  Feenberg  MedaUst)  and  Philippe  Monthoux  (Department  of 
Physics,  UIUC)  used  the  NCSA  Cray  Research  Y-MP  to  carry  out  a  strong  coupling 
(Eliashberg)  calculation  of  the  normal  state  properties  and  Tc  for  the  model  experi- 
ment-based magnetic  interaction  between  quasiparticles... 

Crystallography...  Herbert  Hauptman  (Medical  Foundation  of  Buffalo,  Inc.)  won  a 
Nobel  Prize  in  1985  for  development  of  the  "direct  method"  of  protein  structure  de- 
termination from  X-ray  crystallographic  data.  In  a  collaboration  with  Russ  Miller 
(State  University  of  New  York  at  Buffalo),  Hauptman  and  Miller  have  developed  a 
numerical  approach  that  extends  the  "direct  method"  of  determirung  molecular 
structure  from  X-ray  crystallographic  data  to  larger  molecules,  beyond  its  present 
limit  of  about  100  atoms... 

Pulsar  Searching  and  Discovery  ...  Joseph  Taylor  (Dept.  of  Astrophysical  Sciences, 
Princeton  and  a  1993  Nobel  prize  winner  in  Physics)  and  his  colleagues  used  the  PSC 
C90  to  aid  in  the  search  for  pulsars  by  analyzing  radio-frequency  data  gathered  at  the 
Arecibo  Observatory.  Using  supercomputing  to  sift  through  the  enormous  amount 
of  data  obtained,  this  approach  has  resulted  in  the  discovery  of  20  previously  un- 
known pulsars,  including  five  millisecond  pulsars. 

Accretion  Disks  Around  Black  Holes  ...  John  Hawley  (Dept.  of  Astronomy,  Univ.  of 
Virginia)  has  been  using  supercomputers  for  over  a  decade  to  investigate  the  accre- 
tion disks  which  form  around  black  holes.  Using  the  Cray  Research  Y-MP  and  C90 
at  NCSA  and  PSC,  Hawley  developed  a  new  magnetohydrodynamic  (MHD)  code 
which  enabled  him  and  Steven  Balbus  to  explore  the  nature  of  a  powerful  MHD  in- 
stability in  accretion  disks.  ...  For  this  discovery,  Hawley  received  the  1993  Helen  B. 
Warner  Prize  from  the  American  Astronomical  Society. 

Alpha  Shapes,  Biomolecules,  and  Cosmology  Alpha  shapes,  a  form  of  geometric 
modeling  developed  by  the  1993  Waterman  Award  winner  Herbert  Edelsbruner 
(Dept.  of  Computer  Science,  UIUC)  and  NCSA  staff  member  Ping  Fu,  focuses  on  the 
formal  definition,  construction,  and  measurement  of  shapes  for  any  given  point  set 
in  space.  [They  are...]  useful  in  studying  water  molecules  residing  inside  a  protein. 
NCSA  users  have  discovered  other  related  applications  of  alpha  shapes  by  applying 
them  to  such  diverse  fields  as  adaptive  grid  generation,  medical  image  analysis,  vi- 
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sualizing  the  structure  of  earthquake  data,  and  the  large-scale  structure  of  the  uni- 
verse. 

The  Rise  of  the  Mosaic/WWW  Information  Infrastruture  NCSA  developed  the 
Mosaic  user  interface  software  which  provides  point-and-click  access  to  the  diverse 
information  storage  protocols  of  the  Internet,  such  as  World  Wide  Web  (WWW), 
Gopher,  FTP,  and  WAIS....  In  1994,  NCSA  was  given  Infozvorld's  Publisher's  Indus- 
try Achievement  Award. 

Acronyms 


AFS  Andrew  File  System  (developed  at  Carnegie-Melon  University  and 

named  for  its  benefactors,  Andrew  Carnegie  and  Andrew  Melon) 

ARPA  Advanced  Research  Project  Agency  of  the  Department  of  Defense 

(previously  called  DARPA) 

ASC  Division  of  Advanced  Scientific  Computing,  Computer  and  Informa- 

tion Sciences  and  Engineering  Directorate  (CISE),  NSF 

ATM  Asynchronous  Transmission  Mode  -  high  bandwidth  transmission 

technology  for  WAN  use 

BIMA  Berkeley,  Illinois,  MAryland  telescope 

BLANCA      NSF  Gigabit  testbed  -  see  CASA 

C  A  programming  language  developed  at  Bell  Labs  (probably  the  3rd  in  a 

series)  and  used  as  an  easily  portable  systems  development  language. 

C++  An  Object  Oriented  extension  of  C 

CAD  Computer  Aided  Design 

CAPS  Center  for  Analysis  and  Prediction  of  Storms 

CASA  NSF  Gigabit  testbed  -  see  BLANCA 

CAT  Cornell  University's  Biotechnology  Center  for  Advanced  Technology 

CAVE  Cave  Automated  Virtual  Environment 

CCMS  Computational  Center  for  Macromolecular  Structure  —  an  SDSC, 

UCSD,  and  Scripps  Research  Institute  collaboration 

CD  Compact  Disk 

CFS  Common  File  System 

CMRR  Center  for  Magnetic  Recording  Research 

CNRI  Corporation  for  National  Research  Interests 

CPU  Central  Processor  Unit 

CTC  Cornell  Theory  Center 

CTSS  Cray  Time  Sharing  System 

DASC  see  ASC 

DEC  Digital  Equipment  Corporation 

DICE  Distributed  Collaboration  Environment) 

DJMS  Dynamic  Job  Mix  Scheduler 

DMF  Data  Migration  Facility  (Cray) 

DX  Data  eXpIorer 

EOSDIS  NASA  Earth  Orbitting  Satellite  Distributed  Information  System 
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EPA  Environmental  Protection  Agency 

FAST  Flow  Analysis  Software  Toolkit 

FDDI  Fiber  Data  Distribution  Interface  (a  100  Mps  network  medium) 

FORTRAN  FORmula  TRANslator  -  Earliest  high  level  programming  language  — 
still  in  common  use  on  high  performance  computers 

FTP  File  Transfer  Protocol 

GC  Grand  Challenge 

Giga  One  Billion  -  10^ 

GIS  Geographic  Information  System 

Gopher  Internet  Browser  developed  at  the  University  of  Minnesota  (Golden 
Gophers) 

HDF  NCSA  Hierarchical  Data  Format 

HiPPI  High  Speed  Peripheral  Parallel  Interface 

HP  Hewlett  Packard 

HPC++  An  object  oriented  varient  of  the  C  language  for  High  Performance 
Computers 

HPCC  High  Performance  Computing  and  Communications 

HPF  High  Performance  FORTRAN 

HPSS  High  Performance  Storage  System 

HTTP  Hyper-Text  Transfer  Protocol  for  the  Internet 

IBM  International  Business  Machines 

IFS  AFS  port  by  Univ.  of  Michigan 

InterNIC  Internet  Network  Information  Center 

rUPAP  International  Union  of  Pure  and  Applied  Physics 

IVEM  Intermediate  Voltage  Electron  Microscope 

LANL  Los  Alamos  National  Laboratory 

LAN  Local  Area  Network 

LIGO  Laser  Interferometry  Gravitational  Observatory 

LLNL  Lawrence  Livermore  National  Laboratory 

MacCHESS  Cornell  High  Energy  Synchrotron  Source  for  Macromolecular  Model- 
ing 

MHD  Magneto  HydroDynamics 

MILC  MEMD  Lattice  Calculations  Collaboration 

MIMD  Multiple  Instruction,  Multiple  Data 

Mosaic  Internet  Hypertext  Browser  —  WWW  client  developed  at  NCSA 

MPP  Massively  Parallel  Processors 

MR  A  MetaCenter  Regional  Affiliate 

MRI  Magnetic  Resonance  Interferometry 

NAS  National  Academy  of  Sciences 

NCAR  National  Center  for  Atmospheric  Research 

NCSA  National  Center  for  Supercomputer  Applications 

NIH  National  Institutes  of  Health 

NH  National  Information  Infrastructure 

NOAA  National  Oceanic  and  Atmospheric  Administration 

NRAO  National  Radio  Astronomical  Observatory 

NREN  National  Research  and  Education  Network 
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NSF 

NSL 

PICT 

PSC 

PVM 

QCD 

RAID 

REL 

REU 

RISC 

ROM 

SDF 

SDMIR 

SDSC 

SGI 

SIGGRAPH 

SIMD 

STM 

TCP/IP 

Telnet 

Tera 

TMC 

UNICOS 

UNIX 


vBNS 

VPLA 

VR 

WAIS 

WAN 

WWW 


National  Science  Foundation 
National  Storage  Lab 
A  popular  image  format  for  computers 
Pittsburgh  Supercomputer  Center 
Parallel  Virtual  Machine 
Quantum  ChromoDynamics 
Redundant  Array  of  Inexpensive  Disks 
Renaissance  Experimental  Laboratory 
Research  Experience  for  Undergraduates  (NSF  program) 
Reduced  Instruction  Set  Computer 
Read  Only  Memory 
Standard  Data  Format 

San  Diego  Microscopy  and  Imaging  Resource 
San  Diego  Supercomputer  Center 
Silicon  Graphics,  Inc. 

Special  Interest  Group  -  Graphics  —  an  influential  and  popular  visual- 
ization organization  and  annual  trade  show 
Single  Instruction,  Multiple  Data 
Scanning  Tunneling  Microscope 
Tranmission  Control  Protocol /Internet  Protocol 
Internet  Computer  logon  Services 
One  trillion  -  10l2 
Thinking  Machines,  Corp. 
Cray's  Version  of  Unix 

Popular  portable  computer  operation  system  developed  at  Bell  Labs  - 
reportedly  does  not  stand  for  anything  -  although  some  believe  that  it 
was  a  pun  on  Multics,  a  time  sharing  systems  then  in  use  at  MIT 
very  high  speed  Backbone  Network  Service 
Visual  Programming  Language  for  Arumation 
Virtual  Reality 

Wide  Area  Information  System 
Wide  Area  Networks 

World  Wide  Web  -  a  set  of  commications  protocols  for  Hypertext  estab- 
lished at  CERN. 


XMP  &  YMP  -  versions  of  Cray  computer  systems  ~  vector  multi-processors—  used 
as  references  for  computer  performance 
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Mr.  BOEHLERT.  Thank  you,  very  much. 

Well,  I  am  constrained.  I  will  not  comment.  Dr.  Roskies? 

STATEMENT  OF  DR.  RALPH  ROSKIES,  SCIENTIFIC  DIRECTOR, 
PITTSBURGH  SUPERCOMPUTING  CENTER,  AND  PROFESSOR 
OF  PHYSICS,  UNIVERSITY  OF  PITTSBURGH,  PITTSBURGH, 
PENNSYLVANIA 

Dr.  Roskies.  Thank  you. 

I  am  Ralph  Roskies.  I  am  Scientific  Director  at  the  Pittsburgh 
Supercomputing  Center,  which  is  itself  a  joint  effort  between  Car- 
negie Melon  University  and  the  University  of  Pittsburgh. 

I  am  also  Professor  of  Physics  at  the  University  of  Pittsburgh. 

Mr.  Chairman,  I  agree  with  the  sentiments  that  you  have  heard 
that  the  NSF  Supercomputing  Centers  Program  has  been  ex- 
tremely successful  and  has  helped  the  Nation  retain  its  world  lead- 
ership in  high  performance  computing  and  communications. 

It  has  enabled  very  significant  contributions  to  science  and  engi- 
neering, improved  the  state  of  the  art  in  computing  and  in  elec- 
tronic communication,  and  has  contributed  substantially  to  educat- 
ing the  next  generation  of  scientists  in  the  techniques  that  they 
will  need  to  be  most  productive. 

The  talented  staff  at  the  Pittsburgh  Supercomputing  Center  and 
the  scientists  throughout  the  country  who  have  made  use  of  our  fa- 
cilities have  all  made  major  contributions  to  these  areas. 

We  in  Pittsburgh  have  particularly  championed  the  largest  com- 
putations feasible  with  today^s  technology.  I  would  like  to  cite  just 
a  few  examples  of  advances  in  science  which  we  have  enabled 
which  have  immediate  positive  impact  on  the  quality  of  life  for  our 
fellow  Americans. 

One  of  the  earliest  successes  of  the  program  was  work  done  at 
the  Pittsburgh  Supercomputing  Center  which  led  to  a  better  under- 
standing of  the  generation  of  smog  over  Los  Angeles. 

This  study  has  had  significant  impact  on  California's  regulations 
to  improve  Los  Angeles'  air  quality. 

Now  you  will  hear  in  the  next  panel  from  Dr.  Droegemeier  from 
Oklahoma,  working  with  us  on  the  world's  most  powerful  comput- 
ers that  he  was  able  for  the  first  time  to  make  accurate  predictions 
of  the  course  of  a  severe  thunderstorm  six  hours  ahead  of  time. 

He  will  explain  how  this  will  lead,  among  other  things,  to  safer 
and  more  convenient  and  more  cost-effective  air  travel. 

The  Pittsburgh  Supercomputing  Center  has  led  in  the  application 
of  high-performance  computing  to  biomedicine  and  health  care. 

One  example  of  the  important  work  done  in  this  area.  Professor 
Charles  Peskin  of  the  Courant  Institute  of  New  York  exploited  our 
unique  capabilities  to  develop  a  very  accurate  simulation  of  a  func- 
tioning heart.  This  will  allow  better  design  of  artificial  heart  valves 
and  require  less  experimentation  on  live  subjects. 

The  Pittsburgh  Supercomputing  Center  has  also  made  substan- 
tial contributions  to  improving  and  extending  the  reach  of  elec- 
tronic networks  which  have  now  blossomed  into  the  Internet  and 
the  Information  Superhighway. 

Because  of  our  networking  expertise,  we  were  able  to  design  the 
infrastructure  for  Common  Knowledge  Pittsburgh,  which  was  one 
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of  the  earliest  and  most  highly  praised  pilot  programs  to  transform 
K  through  12  teaching  by  incorporating  computing  technology. 

The  work  and  advice  of  the  Common  Knowledge  team  has  led  the 
Pittsburgh  Public  Schools,  a  major  urban  public  school  system,  to 
undertake  a  five-year,  $75  million  plan  to  extend  this  infrastruc- 
ture throughout  the  entire  school  system. 

I  am  honored  that  the  Pittsburgh  Supercomputing  Center  has 
contributed  substantially  to  these  and  many  other  significant  na- 
tional achievements. 

It  is  very  gratif3ring  that  every  review  of  the  Supercomputing 
Centers  has  found  them  to  be  of  great  success. 

One  of  the  great  strengths  of  the  program  has  been  the  diversity 
of  different  centers'  approach,  and  I  hope  that  this  strength  will  be 
retained  in  the  new  program. 

We  at  the  Pittsburgh  Supercomputing  Centers  and  our  partners 
look  forward  to  the  opportunity  for  making  substantial  contribu- 
tions to  the  flourishing  of  computational  science  and  engineering  in 
the  country  during  the  next  ten  years. 

Thank  you. 

Mrs.  MORELLA  [presiding].  Thank  you.  Dr.  Roskies,  let's  hear 
now  from  Dr.  Pewitt. 

STATEMENT  OF  DR.  DOUGLAS  PEWITT,  ACTING  DIRECTOR, 
SAN  DIEGO  SUPERCOMPUTING  CENTER,  LA  JOLLA,  CALI- 
FORNIA 

Dr.  Pewitt.  Thank  you  for  the  opportunity  to  testify  for  the  San 
Diego  Supercomputing  Center  on  behalf  of  the  NSF  program  for 
supercomputer  centers.  I'm  Chief  Operating  Officer  of  the  SDSC 
Incorporated,  a  subsidiary  of  General  Atomics  that  operates  the 
Center,  and  am  Acting  Director  of  SDSC. 

I  was  involved  in  this  program  at  its  inception.  In  1983,  as  As- 
sistant Director  of  OSTP,  I  chaired  the  FCCSET  Committee  on 
Supercomputing  which  recommended  the  policy  under  which  the 
program  was  developed. 

The  Committee  requested  information  on  accomplishments  of  the 
centers.  The  many  accomplishments  of  SDSC  are  addressed  in 
some  depth  in  my  written  testimony  and  in  other  materials  pro- 
vided. 

Rather  than  dwell  at  length  on  these,  it  is  useful  to  recognize 
that  today  SDSC  embodies  a  vision  of  many  who  saw,  in  the  early 
1980s,  the  need  to  provide  supercomputing  capability  for  research 
universities. 

Some  here  will  recall  that  the  need  was  recognized  particularly 
in  the  Lax  Committee  report  to  the  National  Science  Foundation. 

We  remember  also  at  that  time  there  was  a  call  to  respond  to 
a  Japanese  fifth  generation  computer  initiative  with  a  federal  pro- 
gram to  develop  supercomputers. 

At  OSTP,  we  saw  that  a  FCCSET  Committee  would  be  useful  to 
establish  a  framework  for  federal  policy. 

The  outcome  was  the  determination  that  the  Federal  Govern- 
ment could  best  contribute  by  being  a  good  customer  for  private 
supercomputer  firms. 

And  federal  agencies  were  encouraged  to  provide  supercomputing 
capability  to  their  research  communities. 
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The  accomplishments  of  the  past  ten  years  have  shown  this  pol- 
icy to  have  been  sound.  Purchases  of  state  of  the  art 
supercomputers  based  on  specific  research  needs  has  proven  effec- 
tive. 

We  believe  the  vision  of  the  Lax  Report  has  been  achieved. 

The  NSF  Supercomputer  Center  Programs  continue  to  be  a  wise 
investment  for  our  country,  but  we're  entering  a  new  phase  for 
computational  science.  We  cannot  rest  on  past  accomplishments. 

Competition  is  the  only  way  yet  proven  to  ensure  continuation  of 
a  vigorous  program.  NSF  has  solicited  proposals  for  a  partnership 
for  advanced  computational  infrastructure  program. 

This  program  evolved  from  the  current  one  and  is  aimed  at 
broader  objectives.  It  should  result  in  a  program  more  inclusive  of 
organizations  with  diverse  and  complementary  strengths. 

It  should  provide  a  stronger  infrastructure  that  can  help  the  na- 
tion maintain  scientific  leadership  and  economic  competitiveness. 

Indeed,  as  it  is  clear  that  advanced  computational  technology  is 
central  to  the  information  revolution  itself,  it  is  clear  that  the  new 
challenge  facing  us  is  to  expand  availability  of  this  technology  to 
include  wider  constituencies  within  the  university  systems,  indus- 
try and  the  public. 

For  this  reason,  we  endorse  the  NSF  solicitation.  We  are  con- 
cerned, however,  that  the  language  in  the  solicitation  appears  to 
favor  university  led  proposals.  If  this  were  true,  it  would  bias  the 
competition  against  San  Diego's  Supercomputing  Center  because  it 
is  the  only  one  of  the  four  existing  supercomputing  centers  that  is 
operated  by  industry. 

Given  the  outstanding  record  of  performance  of  the  San  Diego 
Center  under  General  Atomics  leadership,  we  doubt  there  was  any 
intent  to  prejudice  the  competition  against  SDSC. 

And  we  offer  the  expertise  of  SDSC,  documented  in  our  written 
statement,  operated  over  the  last  decade  by  General  Atomics  on  the 
campus  of  UCSD,  under  an  agreement  with  NSF,  as  proof  of  the 
excellence  possible  with  such  a  collaboration. 

We  are  encouraged  by  Dr.  Young's  assurance,  at  this  late  date, 
that  proposals  from  private  industry  will  be  accepted.  But 
preproposals  are  due  in  less  than  four  weeks. 

Universities  have  been  developing  their  preproposals  for  several 
months.  We  urge  that  provision  be  made  to  assure  a  level  playing 
field  for  private  industry  participation. 

Thank  you,  Mrs.  Chairman. 

[The  prepared  statement  of  Dr.  Pewitt  follows:] 
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Presentation  by  N.  Douglas  Pewitt 

ACTING  DIRECTOR  OF  THE  SAN  DIEGO  SUPERCOMPUTER  CENTER 
BEFORE  THE  SUBCOMMITTEE  ON  BASIC  RESEARCH  OF  THE  COMMITTEE  ON  SCIENCE  OF 

THE  U.S.  House  of  Representatives 

with  regard  to  the  accomplishments  of  the  national  science  foundation's 
Supercomputer  Centers'  program 

AND  comments  ON  THE 

Partnership  for  Advanced  Computational  Infrastructure  program 

March  19,  1996 

Mr.  Chairman,  committee  members:  I  thank  you  for  the  opportunity  to  appear 
today  to  testify  on  behalf  of  the  very  successful  National  Science  Foundation  (NSF) 
supercomputer  centers  program.  Although  I  have  only  been  acting  director  at  the 
San  Diego  Supercomputer  Center  a  few  weeks,  I  was  involved  with  this  program 
at  its  inception  and  over  the  years  have  paid  close  attention  to  its  accomplishments 
and  activities. 

My  own  involvement  as  assistant  director  in  the  Office  of  Science  and  Technology 
Policy  (OSTP)  in  the  early  1980s  allowed  me  to  participate  in  the  process  that 
founded  the  centers,  and  I  have  been  pleased  to  see  the  centers  so  clearly  follow 
the  direction  set  by  the  FCCSET  and  NSF  in  implementing  the  program. 

This  subcommittee  has  requested  information  on  the  accomplishments  of  the  NSF 
supercomputer  centers — those  accomplishments  are  addressed  below.  Dr.  Wayne 
Pfeiffer,  SDSC  Deputy  Director  for  Research,  is  with  me  today  to  help  respond  to 
any  questions  you  may  have. 

My  remarks  come  from  my  perspective  on  the  program.  SDSC  is  the  embodiment 
of  a  vision  held  by  many  in  the  early  1980s  concerned  with  ensuring  the  availability 
of  leading-edge  supercomputing  capability  for  the  nation's  research  community,  par- 
ticularly the  academic  research  community.  Some  here  will  recall  that  the  need  for 
such  capability  was  well  recognized,  and  was  documented  in  a  comprehensive  report 
to  NSF  chaired  by  Peter  Lax.  We  also  remember  that  there  was  a  call  for  respond- 
ing to  the  challenge  of  the  Japanese  Fifth  Generation  Computer  initiative  with  a 
Federal  program  to  develop  the  next  generation  of  supercomputers. 

At  OSTP,  we  saw  that  a  FCCSET  Committee  would  be  a  useful  forum  to  establish 
a  framework  for  Federal  policy.  The  outcome  of  that  process  was  the  determination 
that  the  Federal  government  could  best  contribute  by  being  a  good  customer  for  pri- 
vate supercomputing  firms.  To  implement  that  policy.  Federal  agencies  were  encour- 
aged to  allocate  resources  to  provide  supercomputing  capability  to  their  respective 
research  communities. 

We  endorsed  the  model  of  the  Magnetic  Fusion  Energy  (MFE)  program  to  provide 
this  capability  through  centers  such  as  the  MFECC  at  the  Lawrence  Livermore  Na- 
tional Laboratory,  now  the  National  Energy  Research  Supercomputing  Center 
(NERSC).  The  present  director  of  NSF's  Advanced  Scientific  Computing  program 
helped  provide  us,  almost  two  decades  ago,  with  a  successful  model  of  how  such  ca- 
pability could  be  provided  broadly  from  a  central  site. 

The  accomplishments  of  the  past  10  years  have  shown  this  policy  to  have  been 
sound.  Picking  winners  isn't  likely  to  be  successful,  but  purchases  of  state-of-the- 
art  supercomputers  based  on  specific  research  needs  have  proven  effective. 

Now  let  me  return  to  the  successes  of  the  NSF  program.  Working  in  concert,  the 
NSF  supercomputer  centers  have 

•  Enabled  new  science 

•  Developed  higher  performance  computing  technologies 

•  Supported  the  publication  of  thousands  of  papers  in  the  peer-reviewed  scientific  lit- 

erature 

•  Enhanced  educational  opportunities  in  a  wide  variety  of  ways 

•  Modernized  industr/s  design,  engineering  and  manufacturing  processes 

•  Created  spin-off  businesses 

•  Established  a  national  "brain  trust"  with  the  vision  and  experience  to  develop  and 

implement  the  nation's  computational  infrastructure 

To  illustrate  this  record  of  excellence  and  achievement,  let  me  give  some  specific 
details  relating  to  SDSC. 

New  science  enabled  at  SDSC  spans  a  broad  range  of  disciplines.  In  biomedicine, 
an  interdisciplinary  team  of  researchers  from  SDSC,  the  University  of  California  at 
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San  Diego  (UCSD),  and  The  Scripps  Research  Institute  determined  the  structure  of 
a  compUcated  macromolecule  called  the  cAMP-dependent  protein  kinase.  The  signal 
from  this  molecule  is  essential  for  the  function  of  many  hormones,  including  adrena- 
line. This  structure  is  providing  clues  to  the  function  of  the  molecule  and,  further- 
more, is  being  used  as  a  "template"  to  facilitate  determination  of  the  structures  of 
related  kineses,  which  by  latest  count  may  number  in  the  thousands. 

In  another  biomedical  example,  a  researcher  at  the  University  of  Iowa,  who  is  also 
a  practicing  ear  surgeon,  is  modeling  the  human  hearing  process  to  improve  the  de- 
sign of  cochlear  implants  for  hearing-impaired  people,  minimizing  the  need  for  ex- 
perimentation on  animals. 

In  environmental  sciences,  SDSC  researchers  are  developing  a  hydrodynamic 
model  of  San  Diego  Bay  to  help  local  public  policy  makers  evaluate  the  potential 
impact  of  proposed  regulations  on  commercial  and  recreational  use  of  the  bay.  This 
model  will  also  allow  them  to  evaluate  the  effectiveness  of  previously  implemented 
regulations.  This  model  is  based  on  the  collective  efforts  of  more  than  30  monitoring 
organizations  collecting  data  in  the  field.  In  addition,  these  data  will  be  normalized 
and  made  available  on  SDSC's  Web  server  as  a  public  source  of  environmental  infor- 
mation for  other  interested  researchers,  professors,  teachers,  and  students. 

Also  in  environmental  science  we  can  point  to  a  project  conducted  in  collaboration 
with  American  Cyanamid,  a  major  pesticide  manufacturer,  and  the  U.S.  EPA  to  es- 
tablish scientific  criteria  to  predict  pesticide  run-off  over  the  entire  continental  U.S. 
The  results  of  this  project  will  be  used  by  Texas  A&M  University  to  develop  a 
desktop  application  that  will  be  deployed  in  EPA  field  offices.  This  application  will 
run  standard  scenarios  to  determine  the  probability  of  pesticide  leaching  into  a 
source  of  surface  water  by  erosion  or  through  surface  water  run-off. 

A  third  environmental  project,  conducted  by  the  California  Air  Resources  Board, 
studied  smog  patterns  in  the  Los  Angeles  basin.  This  work  led  to  changes  in  the 
State's  regulations  on  auto  emissions. 

In  engineering,  McDonnell  Douglas  is  designing  new  airplanes,  not  through  con- 
ventional and  very  expensive  wind  tunnel  testing,  but  by  conducting  computational 
fluid  dynamics  studies  to  understand  wind  flow  around  the  fuselage  and  wings.  The 
California  Department  of  Transportation  is  studying  the  breakaway  patterns  of 
signs  and  lighting  supports  along  the  highway  to  prevent  additional  damage  to  auto- 
mobiles and  their  passengers  in  a  crash.  An  engineering  firm  in  Sacramento, 
Imbsen  Associates,  is  working  with  SDSC  on  a  computational  retrofit  of  the  Golden 
Gate  Bridge.  The  results  of  this  project  will  be  a  complete  set  of  engineering  plans 
supplied  to  the  Golden  Gate  Bridge  Authority. 

In  astronomy,  SDSC  researchers  performed  the  largest  simulation  to  date  of  col- 
liding galaxies.  This  project,  incidentally,  was  a  collaborative  one  with  NCSA,  which 
visualized  the  simulation  results  and  produced  an  IMAX  movie  Cosmic  Voyage. 
Watch  for  its  debut  at  the  Smithsonian  National  Air  and  Space  Museum  this  spring. 
In  another  astronomy  project,  a  researcher  at  the  Kitt  Peak  National  Observatory 
is  conducting  an  environmental  survey  of  mountain  tops  in  Hawaii  to  help  select 
the  best  site  for  a  new  observatory;  in  addition,  he  is  simulating  the  effect  of  swirl- 
ing winds  inside  the  observatory  on  proposed  types  of  optics. 

The  development  of  new  computing  technologies,  orchestrated  by  the 
supercomputer  centers,  has  led  to  great  advancements  in  researchers'  ability  to  ex- 
plore new  questions  in  their  respective  fields.  Without  these  advances,  many  re- 
search programs  would  be  years  behind  their  current  state.  And  we  have  developed 
them  typically  in  collaboration  with  American  computer  hardware  and  software  ven- 
dors. 

In  this  arena,  SDSC  has  worked  with  two  American  computer  vendors,  Cray  Re- 
search and  Intel,  to  enhance  the  operating  systems  on  their  supercomputers;  these 
enhancements  subsequently  were  incorporated  into  their  commercial  operating  sys- 
tem releases.  SDSC  has  developed  significant  applications  software,  notably  in  the 
context  of  the  Computational  Center  for  Macromolecular  Structure,  which  was  re- 
sponsible for  the  protein  determination  I  mentioned  earlier,  and  the  National  Bio- 
medical Computation  Resource  (NBCR),  which  is  making  parallel  applications  avail- 
able to  researchers  in  biomedicine.  In  addition,  SDSC  is  well  known  for  its  leader- 
ship in  developing  visualization  tools,  notably  the  SDSC  Image  Tools,  used  by  peo- 
ple across  the  country  to  convert  files  among  more  than  30  different  graphics  for- 
mats, and  SDSC  WebView,  a  Web  browser  based  on  the  Virtual  Reality  Modeling 
Language.  SDSC  is  also  developing  the  necessary  software  interfaces  to  make  non- 
traditional  research  capabilities  available  on  the  Internet.  These  include  one  of  the 
few  intermediate-voltage  electron  microscopes  in  the  U.S.  and  a  rapid  prototyping 
device,  which  produces  solid  models  of  3-D  geometry  data  sets.  In  the  case  of  the 
latter,  each  model  produced  to  date  has  revealed  new  features  in  the  data  set  that 
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had  not  been  apparent  from  computer  graphics  displays,  motivating  new  research 
directions. 

SDSC  has  also  experimented  successfully  with  new  paradigms  in  computing. 
Since  supercomputers  are  still  not  fast  and  powerful  enougn,  SDSC  has  worked  with 
researchers  at  the  National  Center  for  Atmospheric  Research  and  UCLA  in  separate 
projects  to  distribute  global  climate  models  across  supercomputers  connected  over 
long  distances  by  high-speed  telecommunications  links. 

Further,  SDSC  has  worked  to  increase  wide-area  networking  speeds  through  im- 
plementation of  a  gigabit-per-second  network  linking  SDSC,  Caltech,  the  Jet  Propul- 
sion Laboratory,  and  the  Los  Alamos  National  Laboratory.  One  of  our  newest  and 
most  exciting  projects  is  a  collaboration  with  IBM  to  integrate  a  massive  data  archi- 
val system  with  an  easy-to-use  database  to  facilitate  more  specific  searches  of  huge 
amounts  of  data.  This  capability  will  enable  what  we  call  data-intensive  comput- 
ing," which  I'll  return  to  in  a  minute. 

SDSC  has  also  had  a  significant  impact  on  education  and  training  in  diverse 
ways.  Hundreds  of  teachers  and  students  participate  in  SDSC  activities,  and  thou- 
sands have  experienced  the  benefits  of  curricular  enhancements  to  their  classroom 
activities  as  a  result  of  SDSC  programs.  SDSC  supercomputer  and  visualization  re- 
sources are  used  in  hundreds  of  university  classes,  and  researchers  with  SDSC  com- 
Euter  accounts  benefit  from  hands-on  training.  SDSC  research  staff  teach  classes  at 
fCSD,  UCSD  Extension,  and  San  Diego  State  University.  In  addition,  10  profes- 
sional staff  at  SDSC  are  enrolled  in  Ph.D.  programs,  and  45  graduate  and  under- 
graduate students  are  working  at  SDSC  in  research,  programming,  technical  writ- 
ing, and  other  positions.  SDSC  has  facilitated  curriculum  development  for  primarily 
undergraduate  students,  has  conducted  a  three-year  program  for  high  school  science 
and  math  teachers  to  help  them  incorporate  computational  methods  into  their  class 
curricula,  and  is  mentoring  a  group  of  teenage  minority  girls  in  a  "Science  Scholars" 
program  to  encourage  them  to  pursue  college  educations  and  careers  in  the  sciences. 

Industry,  too,  has  benefited  from  SDSC  programs.  Through  our  industrial  part- 
nership program,  some  35  companies,  including  large  aerospace,  pharmaceutical, 
and  banking  companies,  are  learning  to  adapt  computational  methods  to  their  de- 
sign, engineering,  and  manufacturing  processes.  In  addition,  SDSC  has  spun  off  at 
least  three  private  ventures  in  archival  storage,  Internet  service  provision,  and  vis- 
ualization software  for  molecular  modeling. 

One  of  the  most  important,  and  often  underestimated,  facets  of  SDSC  is  the  brain 
trust  we  have  built  up  over  our  10-year  history.  This  group  of  125  researchers,  soft- 
ware development,  and  user  and  operations  support  staff — along  with  the  staffs  at 
the  other  centers — is  providing  a  national  vision  for  development  and  implementa- 
tion of  the  nation's  computational  infrastructure,  both  in  support  of  science  and  en- 
gineering and  in  support  of  computing  generally  for  the  citizenry. 

SDSC  has  furthered  and  strengthened  our  scientific,  technology  development,  and 
educational  activities  by  providing  leadership  in  organizing  and  hosting  major  pro- 
fessional society  conferences  and  meetings  at  which  current  and  future  projects  are 
discussed.  These  meetings  include  SIGGRAPH  91,  the  Cray  User  Group,  the  Intel 
Supercomputer  User's  Group,  SUPERCOMPUTING  '95,  and,  more  recently,  meet- 
ings on  Internet  statistics,  data  management,  and  biodiversity. 

SDSC,  as  an  organization  and  in  terms  of  the  efforts  of  individual  staff  members, 
is  active  in  community  service.  Many  of  you  will  probably  remember  fugitive  com- 
puter criminal  Kevin  Mitnick  who  about  one  year  ago  was  apprehended  by  the  FBI 
because  of  the  technical  expertise  provided  by  SDSC  staff.  In  addition,  as  part  of 
our  recent  hosting  of  SUPERCOMPUTING  '95,  our  networking  staff,  with  help  from 
the  other  centers,  completely  wired  the  San  Diego  Convention  Center  for  tele- 
communications. You'll  note  that  this  is  the  site  of  the  upcoming  Republican  Na- 
tional Convention.  SDSC  also  provided  a  mirror  Web  server  to  support  electronic  de- 
mocracy in  action  through  Vice  President  Gore's  electronic  town  meeting  held  last 
May,  provided  the  Internet  video  broadcast  of  the  signing  of  the  Telecommuni- 
cations Act  last  month,  and  a  couple  of  weeks  ago  participated  in  NetDay96,  a 
grass-roots  effort  to  wire  schools  in  California. 

Future  developments  at  SDSC  that  we  can  look  forward  to  with  great  excitement 
include  our  strategic  emphasis  on  designing  and  implementing  a  new  paradigm  in 
computing,  called  "data-intensive  computing,"  which  I  mentioned  a  minute  ago. 
Data-intensive  computing  will  be  enabled  by  integrating  multiple-petabyte  archival 
storage  systems  with  easy-to-use  database  interfaces  and  high-speed  telecommuni- 
cations links.  It  will  build  on  SDSC's  long-standing  capabilities  supporting  the  solu- 
tion of  computation-intensive  problems.  This  new  capability  will  facilitate  more  com- 
prehensive studies  in  such  diverse  areas  as  global  climate  modeling,  comparisons 
among  macromolecular  structures,  and  coupling  models  describing  the  electrical  ac- 
tivity and  fluid  dynamics  in  the  human  heart  for  a  more  comprehensive  understand- 
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ing  of  its  function.  In  addition,  it  will  support,  for  the  first  time,  the  integration  of 
real-time  feeds  of  satellite  data  into  a  running  computation  to  produce  more  accu- 
rate and  comprehensive  solutions  to  environmental  problems  based  on  the  most  cur- 
rent data.  This  capability  will  also  support  "data  mining,"-seeking  correlations 
across  multiple  data  sets.  Data  mining,  for  example,  will  be  used  to  enhance  medical 
diagnosis  in  that  it  will  make  it  possible  for  medical  researchers  to  look  for  clues 
to  the  onset  of  disease  by  analyzing  huge  databases  of  medical  records. 

As  you  can  see,  Mr.  Chairman  and  committee  members,  the  NSF  supercomputer 
centers  program  continues  to  be  a  wise  investment  in  research  and  education  for 
our  country.  And  now,  as  a  result  of  the  Hayes  committee  findings,  we  are  entering 
a  new  phase  for  advanced  computational  science.  To  broaden  the  impact  of  the  Fed- 
eral government's  investment  in  computational  science,  NSF  has  issued  a  call  for 
proposals  to  respond  to  the  Partnerships  for  Advanced  Computational  Infrastructure 
program.  This  program  is  a  natural  evolution  of  the  current  one,  strengthening 
NSF's  commitment  to  an  ever-broadening  program  and  increased  access  to  resources 
and  expertise  for  the  scientific  community. 

The  new  solicitation  is  having  an  impact  in  many  ways,  but  the  primary  impact 
is  the  solidification  of  our  partnerships,  both  existing  and  future,  in  a  program  to 
address  the  computational  needs  of  the  country.  Long-established  partners  are  eager 
to  share  resources  with  new  partners  in  this  effort.  And  new  partners,  most  of 
whom  have  come  forth  on  their  own  seek  to  join  this  expanding  collaboration. 

We  welcome  this  broader  program.  It  will  be  more  inclusive  of  organizations  with 
diverse  and  complementary  strengths  in  groundbreaking  areas  of  research.  And,  ul- 
timately, it  will  result  in  a  stronger  infrastructure  that  can  help  the  nation  main- 
tain world  leadership  in  scientific  progress  and  economic  development. 

In  conclusion,  we  believe  the  early  vision  held  in  the  Lax  report  has  been 
achieved.  The  day-to-day  effort  necessary  to  accomplish  grand  ideas  has  been  taking 
place  at  the  supercomputer  centers  for  10  years,  and  the  credit  belongs  to  those  who 
have  achieved  the  vision.  They  deserve  recognition  for  accomplishing  what  seemed 
a  great  challenge  in  1982. 

But  we  cannot  rest  on  past  accomplishments.  Competition  is  necessary  and  the 
only  way  yet  proven  to  ensure  a  continuation  of  a  vigorous  program. 

The  reality  of  resources  that  can  be  provided  today  are  different  from  that  of  the 
period  when  the  centers  were  established.  Difficult  as  it  is,  choices  now  need  be 
made,  and,  unfortunately,  this  means  winners  and  losers.  Our  view  is  that  this  is 
healthy.  The  nation  should  fund  excellence,  and  fair  competition  on  a  level  playing 
field  is  the  best  way  yet  devised  for  allocation  of  always  limited  resources. 

We  endorse  the  NSF  solicitation  but  encourage  an  ever  more  comprehensive  com- 
petition. We  believe  there  is  a  legitimate  role  for  private  industry  in  academic 
supercomputing.  We  offer  the  experience  of  SDSC,  documented  in  our  written  state- 
ment, operated  by  General  Atomics  on  the  campus  of  UCSD  under  a  cooperative 
agreement  with  NSF  for  the  last  decade,  as  proof  of  the  excellence  possible  with 
such  a  collaboration. 

Thank  you  again,  Mr.  Chairman,  for  the  opportunity  to  speak  today. 

For  additional  information,  contact: 

Doug  Pewitt 

San  Diego  Supercomputer  Center 

10100  Hopkins  Drive 

La  Jolla,  CA  92093-0505 

619-534-5000 

pewitt@sdsc.edu 

Mrs.  MORELLA.  Thank  you,  Dr.  Pewitt.  We  hope  to  get  some  com- 
ments on  the  challenge  that  you  mention. 

You  all  display  great  enthusiasm  and  commitment  to  the  centers 
and  basically  to  the  new  program. 

I  have  one  question  I'd  just  like  to  direct  to  all  of  you,  and  then 
I'll  defer  to  my  colleagues. 

Dr.  Hayes  spoke  of  the  need  to  provide  centers  with  the  ability 
to  continue  to  recapitalize  their  equipment  periodically,  three  to 
four  years,  or  five  to  seven  years,  in  order  to  stay  in  the  leading 
edge,  which  would  be  ten  to  100  times  faster  than  the  mid-range 
machines. 
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I  just  wondered,  could  you  briefly  describe  your  history  of  recapi- 
talization, you  know,  both  the  time  and  the  dollars.  You  can  start 
at  any  point. 

Maybe  Dr.  Kalos,  you'd  like  to  begin  that,  sir. 

Dr.  Kalos.  Well,  like  the  other  centers,  we  have  recently  recapi- 
talized. Historically,  at  least  through  most  of  our  history,  the  main 
computing  power  at  our  center  were  in  IBM  supercomputers. 

About  two  years  ago,  we  began  to  take  delivery  of  the  new  IBM 
scalable  machines,  and  we  were  the  first  to  receive  a  substantial 
SP-I  and  the  first  to  receive  a  very  large  SP-II. 

These  machines  are  scalable  also  in  the  sense  of  technology.  Not 
only  do  they  scale  in  size  and  memory  computing  power,  but  in  fact 
they  are  designed  so  that  many  of  the  components,  the  processors 
and  the  memory,  can  be  taken  out  and  put  in. 

So  some  of  the  advance  in  technology  does  not  require  completing 
replacing  the  machine. 

Nevertheless,  in  the  technology  that  is  used  here  in  common  with 
the  scalable  machines  at  all  the  centers,  is  one  which  is  on  an  expo- 
nential growth  of  about  a  factor  of  two  every  year  and  a  half.  That 
translates  into  a  factor  of  ten  every  five  years.  That  means  that  the 
machines  we  have,  but  also  the  mid-range  machines  become  obso- 
lete within  a  few  years. 

The  picture  you  may  have  drawn,  that  our  machines  need  to  be 
recapitalized  and  the  mid-range  machines  do  not,  doesn't  apply.  All 
of  the  machines  must  be  recapitalized. 

Now  that  puts  a  strain  on  the  budget.  On  the  other  hand,  it  also 
puts  the  NSF  supercomputing  program  and  the  NSF  scientific  ef- 
fort at  the  forefront  of  the  world. 

I've  traveled  the  world,  I've  visited  centers.  Our  current  program 
is  the  envy  of  every  country  I  have  ever  visited,  and  I  think  we 
need  to  continue  that. 

Mr.  Sensenbrenner.  Dr.  Smarr? 

Dr.  Smarr.  This  is  a  more  complicated  business  than  it  might 
appear  at  first  glance. 

We've  gone  through  what  I  said  before  is  a  phase  change.  And 
one  of  the  major  changes  that  I  think  is  the  biggest  change  in  50 
years  of  designing  supercomputers  is  that  previous  to  this  point,  a 
supercomputer  was  built  by  building  a  processor  that  was  unique 
to  that  supercomputer  and  no  one  bought  that  processor  unless 
they  bought  that  supercomputer.  And  the  supercomputers  that 
were  bought  that  were  in  the  maybe  hundreds  worldwide,  so  that 
was  the  installed  base,  and  therefore  there  was  very  little  install 
base  in  hardware  to  drive  software. 

The  capital  cost  of  the  R&D  required  to  make  those  processors 
and  to  push  all  areas  of  technology  were  so  high  that  we  would 
typically,  up  until  for  the  last  ten  years,  keep  a  machine  for  three 
to  five  years,  and  then  get  another  machine.  And  wherefore  we 
could  pay  out  the  $20,-  or  $30  million  cost  of  that  machine  over 
that  period  of  time. 

What  has  happened  that  has  totally  turned  this  upside  down  is 
that  with  the  advent  of  microprocessors,  that  are  nearly  as  fast  as 
these  special  processors,  but  maybe  one-eighth  to  one-tenth  the 
cost,  because  they  have  a  worldwide  install  base  of  hundreds  of 
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thousands  or  even  millions  in  the  work  stations,  and  then  you  sim- 
ply sew  those  together  basically  to  create  the  supercomputer. 

The  capital  costs  have  gone  from,  are  much  lower  in  a  sense,  but 
because  they're  on  the  work  station  upgrade,  we  have  to  replace 
them  now  about  every  year  to  18  months. 

So  there's  no  machine  that  I  have  that  hasn't  been  upgraded  in 
the  last  six  months,  and  that  I'm  not  planning  to  upgrade  in  the 
next  six  months.  It's  worse  almost  than  having  personal  computers. 

Furthermore,  when  you  look  at  the  goal  that  we  have  of  getting 
Teraflop  machines  before  the  end  of  the  century,  it's  not  so  much 
that  it's  a  Teraflop  machine,  maybe  500  or  1000  of  the  state-of-the- 
art  work  stations,  it's  10,000  times  the  memory  you  have  on  prob- 
ably any  personal  computer  within  half  a  mile  of  my  voice. 

And  the  memory  has  actually  turned  out  to  be  over  half  the  cost 
of  these  machines  but  yet  it's  that  large  memory  in  which  you  place 
the  problem  you're  trying  to  solve. 

So  if  you  want  to  solve  the  Chesapeake  Bay  in  detail  that  in- 
cludes the  ecological  systems  and  the  biosystems  and  the  chemical 
pollutants,  as  well  as  the  tide  flows  and  the  interaction  with  the 
Atlantic,  which  is  what  one  of  our  projects  is  doing,  you  need  that 
kind  of  memory.  And  that  memory  is  not  going  down  in  cost  the 
way  that  the  processors  speeds  are  going  down  in  cost. 

And  there's  a  lot  of  reasons  for  that,  if  you  look  at  the  worldwide 
RAM  market  and  so  forth. 

So  increasingly,  the  money  you're  having  to  spend  is  going  to  the 
memory,  not  the  processors.  And  I  think  that's  a  point  that's 
missed  very  often. 

Furthermore,  you  have  to  spend  a  lot  of  money  to  get  the  data 
in  and  out  of  this  machine  because  you're  creating  so  much  of  it 
so  fast.  This  is  the  so-called  10  bottleneck. 

So  these  machines  are  becoming,  it's  getting  very  difficult  to 
spend  the  kind  of  money  we  used  to  spend  and  spend  three  to  five 
years  pa3dng  it  off. 

Now  we  have  to  pay  it  off  maybe  within  a  year  or  18  months, 
and  that  is  one  of  the  things  that  is  causing  a  crisis. 

The  other  thing  that  was  mentioned  earlier,  I'll  just  make  that 
it's  very  important  to  understand.  I  don't  believe,  certainly  our  cen- 
ter would  not,  and  I  think  I  can  say  this  for  almost  all  the  centers, 
would  not  have  been  able  to  get  the  large  machines  that  we  got, 
the  massive  parallel  machines  with,  say,  500  processors,  without 
ARPA's  financial  investment  in  the  centers. 

If  we  had  only  had  the  dollars  that  were  available  in  the  NSF 
program,  we  would  have  had  to  simply  spread  out  in  time  much 
later  getting  those,  and  the  advantage  to  the  country  of  having  the 
early  access  would  have  been  lost. 

Now  ARPA's  mission  is  moving  onward.  Their  budget  is  under 
tremendous  pressure.  And  I  do  not  believe  that  we  will  see  that 
kind  of  help  from  ARPA  in  the  future. 

This  means  NSF  has  to  share  the  whole  cost,  compared  to  what 
it  was  doing  over  the  last  five  years. 

So  these  two  factors  of  having  to  replace  the  machine  over  a 
much  shorter  time,  and  ARPA's  not  being  there  to  help  share  the 
costs  are  what  are  making  it  more  difficult  to  maintain  our  ma- 
chines at  the  leading  edge. 


177 

Mrs.  MORELLA.  A  very  good  response.  You  knew  to  whom  you 
were  speaking  when  you  mentioned  the  Chesapeake  Bay  as  one  of 
your  examples.  I  appreciated  that  and  the  benefit  of  ARPA. 

[Laughter.] 

Mrs.  MoRELLA.  There  are  two  others  who  are  going  to  give  brief 
responses  to  that  question. 

Dr.  ROSKIES.  We've  had  six  major  supercomputers  at  Pittsburgh 
Supercomputing  Center  since  we  started  ten  years  ago,  and  the 
seventh  is  beginning  to  be  dehvered  this  week., 

What  we've  seen  is  that  the  nominal  price  of  these  machines  has 
roughly  stayed  constant.  But  I  agree  with  what  Ed  Hayes  said, 
that  when  we  started,  we  assumed  that  we  would  pay  off  a  ma- 
chine in  five  years  and  now  we  see  that  a  machine  generation  time 
is  more  like  two-and-a-  half  years.  So  that  we  have,  even  at  the 
same  cost  of  machines,  an  annual  capitalization  cost  which  is  twice 
as  large. 

I  want  to  make  one  comment  about  ARPA.  I  think  ARPA's  help 
was  absolutely  essential,  as  Larry  said,  in  enabling  us  to  get  scal- 
able machines  of  large  size. 

On  the  other  hand,  I  think  also  we  should  recognize  that  that  sit- 
uation was  to  some  extent  anomalous  because  we  were  going 
through  a  major  transition  in  the  technology  of  supercomputing. 

And  what  ARPA  enabled  us  to  do  was  to  fund  and  run  at  the 
same  time,  two  major  supercomputers,  the  traditional  Vector  One, 
and  the  more  experimental  one  that  ARPA  was  funding. 

There's  a  sense  in  which  we've  passed  that  hurdle,  and  I  think 
wnat  you'll  see  at  each  of  the  centers  now  is  not  a  need  for  running 
two  major  very  different,  or  capitalizing  two  major  very  different 
machines. 

So  I  just  wanted  to  put  that  qualification  in  the  record. 

Dr.  Pewitt.  Rather  than  to  address  the  history  of  San  Diego 
Center,  I  think  we  can  go  back  to  something  more  fundamental. 

And  as  we  all  know,  there's  something  called  Grove's  Law  about 
doubling  time  being  18  months  for  processing  power,  and  if  you're 
not  on  the  edge,  you're  not  super  computing. 

That  phenomenon  is  with  us  here.  I  think  three  to  five  years  is 
probably  too  long,  given  the  reality  of  how  fast  the  technology  is 
losing,  and  if  you  want  to  think  about  technology  in  a  way,  if  we 
think  about  the  most  important  computation  perhaps,  at  least  pub- 
licly, that's  done  in  the  last  18  months,  we  take  linking,  what  was 
it,  2600  desktop  machines  over  the  internet  and  cracking  the  clip- 
per chip. 

This  is  supercomputing  too.  There's  a  relationship  here  between 
bandwidth,  how  fast  you  can  shove  the  lines. 

Larry  mentioned  how  fast  that  had  gone. 

We're  going  to  see  bigger  improvements.  In  the  new  infrastruc- 
ture, we're  looking  at  ability  to  link  machines  at  speeds  that  are 
just  unthinkable  a  few  years  ago. 

And  the  problem  of  stajring  up  with  where  supercomputers  are 
is  not  going  to  be  easier.  The  delightfulness  of  it  is  it's  going  to  get 
tougher  and  a  five-year  replacement  cycle  is  just  out  of  the  conceiv- 
able range  of  where  we  ought  to  be  in  this  country  if  we're  going 
to  continue  to  lead  like  we  have  in  the  past. 

Mr.  SCHIFF  [Presiding].  Mr.  Doyle? 
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Mr.  Doyle.  Thank  you,  Mr.  Chairman. 

Dr.  Kalos,  either  you  or  your  twin,  I  beUeve,  sat  across  from  me 
on  the  5:30  USAir  flight  from  Pittsburgh  yesterday. 

Was  that  you? 

Dr.  Kalos.  Yes. 

Mr.  Doyle.  I  wish  I  had  known  that  on  the  way  up.  We  would 
have  had  an  interesting  flight. 

[Laughter.] 

Mr.  Doyle.  But  when  you  walked  in  to  day,  I  said,  I  know  that 
face  from  somewhere  and  it  just  occurred  to  me  from  where. 

Dr.  Kalos.  I  wish  I'd  known  it  too. 

Mr.  Doyle.  I  have  two  questions. 

In  your  written  remarks  and  in  Dr.  Roskies'  written  statements 
also,  you  talked  about  the  decrease  in  collaboration  amongst  the 
centers  as  a  result  of  the  recompetition. 

And  I'm  wondering,  you  know,  how  do  we  deal  with  that  over  the 
next  18  months  of  the  program,  and  I'd  like  each  of  you  maybe  to 
speak  to  that  because  that  can  be  a  problem  and  I  think  it  needs 
to  be  addressed. 

And  secondly,  two  of  you  are  going  to  be  leading  edge  centers 
pretty  soon,  and  two  of  you  aren't.  And  I'd  like  you  to  tell  us  what 
the  future  holds  for  you  if  you're  not  one  of  the  leading  edge  cen- 
ters selected. 

Dr.  Kalos.  As  to  the  first  what  will  happen  to  the  spirit  of  co- 
operation and  its  lack  for  the  next  18  months,  I  think  we  are  sim- 
ply going  to  have  to  live  with  that.  That's  a  fact  of  life. 

I  share  Ralph's  belief  that  that  sense  will  return.  I'm  sure  it  will 
be  nurtured  by  the  NSF  after  the  competition  is  over  and  the  win- 
ners make  friends  again. 

As  for  what  the  future  holds  for  us,  well,  of  course,  we're  quite 
confident  of  winning.  And  I  can  speculate  about  what  the  future 
might  hold  for  a  center  that  is  not  successful. 

And  first  and  foremost,  I  think  there  is  going  to  be  a  tragic  dis- 
sipation of  great  professional  strength,  dedication  that  has  built  up 
at  every  one  of  these  centers.  And  I  think  that  that,  in  and  of  itself, 
will  be  a  loss  for  the  country. 

Dr.  Smarr.  Well  I'm  the  person  who  coined  the  term 
"metacenter."  And  I  spent  a  lot  of  effort  trying  to  build  up  the  spir- 
it of  cooperation  between  the  four  NSF  centers,  along  with  my  col- 
leagues. We  had  many  airport  meetings  lasting  many  hours  and  I 
think  we  learned  how  to  manage  a  national  scale  system  in  the  last 
five  years. 

But  I  think  it's  often  missed  that  during  that  same  time,  a  lot 
of  other  partnerships  and  virtual  teamings  arose  as  well.  And  a 
good  example  I  would  give  of  one  of  the  metacenter  regional  alli- 
ances that  Dr.  Hayes  mentioned  that  program,  one  of  them  is  at 
the  University  of  Illinois  at  Chicago  at  the  Electronic  Visualization 
Laboratory. 

Now  this  is  a  computer  science  £ind  computer  art  center.  It  has 
been  working  for  two  decades  on  fundamental  developments  in  vir- 
tual environments. 

They  were  working  on  the  gloves  back  in  the  mid-seventies,  long 
before  anybody  ever  heard  of  virtual  reality. 
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And  we  have,  by  partnering  with  them,  been  able  to  bring  some 
of  the  most  advanced  virtual  environments  capability  to  the  na- 
tional scientific  community  that  exist  anywhere  in  the  world  today. 
The  Cave,  the  new  Infinity  Wall,  the  Immersadesk. 

We  did  not  develop  that  technology  inside  of  our  center.  It  was 
developed  in  one  of  these  wonderful  bottom  up  creative  acts  that 
happen  all  over  our  country. 

But  because  we  were  able  to  partner  with  them  and  in  fact  in 
terms  of  virtual  teaming,  I  named  the  director  of  that  lab  my  Vir- 
tual Associate  Director  for  Virtual  Environments  while  he  main- 
tained himself  as  director  of  that  center,  right. 

And  so  this  is  sort  of  one  of  those  things  I  read  in  an  airport  book 
on  management  that  I  thought  I'd  try  out. 

It's  worked  wonderfully  and  I  think  it's  a  sign  of  where  we're 
going  in  the  future,  so  I  would  just  answer  precisely  your  question. 

That  I  feel  very  badly  and  I  predicted  this  would  happen  from 
the  beginning,  that  such  a  competition  as  we  have  necessarily 
means  a  lessening  of  the  cooperation  between  the  four  centers. 

On  the  other  hand,  I  guess  I  would  say  honestly  in  the  last  few 
months,  I  have  seen  a  great  increase  in  the  cooperation  with  other, 
smaller  units  throughout  the  country.  And  I  would  be  hardpressed 
to  say  on  balance  whether  the  country  is  better  or  worse  off. 

But  there  are  both  pluses  as  well  as  minuses  going  on. 

As  far  as  what  would  happen  to  the  two  centers  that  will  be 
phased  out  if  that  were  to  be  the  outcome  of  the  solicitation,  first 
of  all,  I  think  it's  very  important  to  remember  that  the  National 
Science  Foundation  has  overseen  exactly  that  process  five  years 
ago  in  which  the  John  Von  Neumann  Center,  one  of  the  original 
five  centers,  was  in  fact  very  effectively  and  efficiently  phased  out 
in  a  very  short  period  of  time. 

All  of  the  users  were  transferred  to  other  centers,  and  we  did  a 
lot  of  cooperation  between  the  centers  to  make  that  happen.  A 
number  of  the  staff  went  to  other  centers.  They  went  out  into  in- 
dustry. 

And  I  think  that  a  similar  process  will  occur. 

I  would  not  care  to  predict  what  would  happen  to  NCSA,  wheth- 
er it  would  become  an  Illinois  State  center  or  just  be  closed.  I  really 
don't  know  that  that  has  been  really  thought  through  very  care- 
fully. 

But  the  NSF  has  done  it  before.  I  think  it  can  do  it  again.  And 
I  think  if  you  look  in  the  private  sector,  we  are  constantly  having 
associations,  start-up  companies,  if  you  like,  come  and  go.  And  peo- 
ple repopulate  and  refertilize  the  overall  environment.  That's  just 
part  of  the  way  that  the  market  works,  and  I  think  it's  the  way 
the  NSF  has  shown  it  is  capable  of  working. 

Almost  every  program  today  that  is  funded  by  the  NSF  wasn't 
funded,  you  know,  say  ten  years  ago.  And  the  ones  that  were  fund- 
ed ten  years  ago  mostly  are  not  funded  today. 

So  to  me,  it's  just  a  part  of  the  breathing  exercise  that  NSF  tra- 
ditionally goes  through.  As  long  as,  as  I  say,  merit  review  is  the 
deciding  mechanism  for  how  that  process  goes  forward. 

Dr.  ROSKIES.  To  answer  your  first  question,  I  think  I  agree  with 
Mai  that  the  question  of  cooperation  between  the  centers  or  the  dif- 
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faculty  of  cooperation  between  the  centers  is  a  fact  of  life  for  the 
next  18  months  until  the  recompetition  is  settled. 

On  the  question  of  what  would  happen,  I  think  it's  interesting  to 
pick  up  on  Larry's  example  of  the  John  von  Neumann  Center  be- 
cause it  is  an  example  of  a  case  where  NSF  has  successfully  phased 
out  a  center. 

However,  if  you  had  done  a  poll,  I  think,  at  the  very  beginning 
of  the  NSF  centers  program,  I  would,  I  think  just  from  the  list  of 
institutions  associated,  that  was  almost  surely  the  most  prestigious 
group  of  institutions  running  an  NSF  supercomputing  center. 

If  there  had  been  only  two  centers,  and  the  von  Noiman  Center 
had  failed,  I  think  it  would  have  been  devastating  to  the  program. 

One  of  the  things  that  helped  that  transition  was  the  fact  that 
there  were  five  centers.  And  I  think  that  part  of  what  I  think 
you're  asking  is  whether  or  not  even  two  centers  might  be  too  few. 
I  think  two  centers  is  still  putting  too  much  of  one's  eggs  in  a  sin- 
gle basket. 

Dr.  Pewitt.  Let  me  answer  those  two  questions  a  little  dif- 
ferently from  a  different  perspective. 

Cooperation,  what  happens  to  it  during  the  competition?  After 
more  than  20  years  of  being  closely  tied  to  science  policy  of  the  fed- 
eral government,  if  you  don't  compete,  you  lose  vigor.  The  only  way 
to  maintain  vigor  in  this  program  is  to  have  periodic  competitions. 

Having  spent  a  lot  of  my  time  in  industry,  I  know  it's  possible 
to  compete  and  cooperate.  The  scientific  collaboration  that  we  have 
we  don't  think  should  necessarily  be  damaged.  Of  course,  we're 
going  to  compete  to  win  and  we're  going  to  keep  our  proposal  strat- 
egies held  tight,  and  we're  not  going  to  cooperate  on  that. 

With  regard  to  the  future,  what  happens.  Number  one,  the  talent 
doesn't  disappear.  The  resources  don't  disappear.  New  work  can  be 
found  if  a  center  is  not  left.  I  share  the  concern  when  you  get  down 
to  the  statistics  of  extremely  small  numbers,  you're  much  more  vul- 
nerable to  collapse.  Three  is  better  than  two.  Two  is  infinitely  bet- 
ter than  one.  And  we  all  share  that. 

And  if  there  are  resources  available,  we'd  all  like  to  see  more 
centers.  When  we  originally  conceived  this  program,  we  thought 
that  there  was  going  to  be  16,  but  budget  realities  are  different 
now  than  they  were  in  the  early  eighties. 

And  I  like  16  better  than  I  like  four.  But  here  we  are. 

Mr.  SCHIFF.  The  gentleman's  time  has  expired. 

Mr.  Doyle.  Thank  you. 

Mr.  SCHIFF.  Mr.  Gutknecht? 

Mr.  Gutknecht.  Thank  you,  Mr.  Chairman. 

I'd  like  to  sort  of  come  at  this  a  little  different  way. 

I  was  in  the  state  legislature  in  Minnesota  and  the  University  of 
Minnesota  has  a  supercomputer  center  not  affiliated  with  NSF. 

One  of  the  questions  I  have  for  each  of  you,  somehow  we  once 
in  a  while  got  drug  into  this  debate  in  the  state  legislature,  was 
exactly  how  they  allocated  time. 

And  you've  raised  the  issue  of  ARPA  and  that  was  a  politically 
sensitive  issue  back  there.  As  a  matter  of  fact,  I  came  to  the  conclu- 
sion that  there  were  certain  members  of  the  faculty  who  were  con- 
stantly trying  to  undermine  the  whole  supercomputing  effort  sim- 
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ply  because  they  were  accepting  contracts  for  research  from  the  De- 
partment of  Defense  to  be  done  on  the  computers  back  there. 

So  the  first  question  I  guess  I  have  for  you  is  how  do  you  allot 
time,  and  how  do  you  avoid  those  kinds  of  barbed  wire  type  entan- 
glements? 

Dr.  Kalos.  The  allocation  of  access  to  our  supercomputer  center 
is  governed  by  a  national  allocation  policy.  And  in  fact  that  com- 
mittee, National  Allocation  Committee,  that  committee  is  in  some 
sense  at  arm's  length  from  me,  so  that  political  questions  really 
don't  arise. 

The  judgments  are  entirely  on  the  basis  of  the  scientific  merit  of 
the  research  and  the  balance  of  resources  required  to  do  the  re- 
search. 

The  most  difficult  part  of  the  Committee's  work  is  to  judge 
whether  some  work  in  astrophysics  deserves  an  equal  amount  of 
computer  time  as  compared  with  some  work  in  biochemistry. 

Dr.  Smarr.  Pittsburgh  and  NCSA  have  a  long  history  of  cooper- 
ating by  having  a  joint  allocation  board.  We  are  the  only  two  cen- 
ters that  do  that.  That  board  meets  quarterly  and  therefore  if  you 
are  in  any  university  in  the  country,  and  you  have  a  good  idea,  you 
can  submit  it  up  to  four  times  a  year  in  principle,  although  in  prac- 
tice typically  it's  done  once. 

Again,  if  you  are  a  university  researcher  and  that  allocation  is 
granted,  then  you  have  access  to  that  time.  You  are  not  charged 
for  it.  You're  not  charged  to  your  grant  because  the  genius  to  me 
of  the  NSF  program  was  that  the  machine  is  paid  for  up  front  by 
the  Foundation  and  an  allocation  is  divorced  from  your  ability  to 
buy  the  time  but  rather  allocated  according  to  the  merit  of  your 
idea  as  judged  by  your  colleagues  through  a  peer  review  system. 

And  recently  in  the  last  year  or  so,  we  have,  as  all  four  centers, 
come  together  to  form  a  national  single  review  board  for  very  large 
allocations  because  the  equivalent  cost,  if  you  were  to  really  figure 
out  how  much  that  is  in  dollars,  is  sufficiently  large  that  it  was  felt 
the  national  review  process  is  needed. 

Furthermore,  some  of  the  very  largest  users  require  several  of 
the  centers.  So  one  PI,  one  principal  investigator,  may  actually  use 
the  architecture  of  say  the  SP-II  at  Cornell  and  the  Vector  machine 
at  Pittsburgh,  and  there's  a  lot  of  things  that  are  built  into  the  sys- 
tem to  avoid  people  dipping  twice  for  the  same  science  and  that 
sort  of  thing. 

I  think  all  reviews  of  this  process  have  been  extremely  favorable. 
In  fact,  I  can't  remember  in  all  of  ten  years  really  a  single  negative 
comment  about  the  allocation  process.  I  think  it's  actually  become 
a  model  for  many  other  places. 

As  far  as  your  second  question  goes,  while  I  was  on  other  cam- 
puses in  1968,  at  the  University  of  Illinois,  the  ILIAC  4  project  was 
being  designed.  This  was  the  world's  first  major  parallel 
supercomputer,  in  today's  dollars  probably  $100  million  project. 
And  it  was  funded  through  the  Defense  Department. 

That  machine  was  never  brought  to  campus  because  of  the  riots 
and  the  threat  to  the  physical  existence  of  this  computer  by  radical 
elements  on  the  campus  was  sufficiently  dangerous  that  it  was  ac- 
tually constructed  behind  barbed  wire  at  NASA-Ames  in  California. 
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Times  have  changed.  And  when  the  ARPA  helped  us  bring  the 
first  large-scale  massively  parallel  production  system  into  the  cen- 
ter's program,  the  CM-5,  which  was  the  supercomputer  in  Jurassic 
Park,  that's  all  the  blinking  lights  in  the  movie,  that  was  met  with 
absolute  joy  by  the  scientific  community  on  our  campus,  as  well  as 
elsewhere. 

And  in  fact  I  have  never  heard  any  negative  remarks  by  any  fac- 
ulty, either  at  our  campus  or  nationally,  and  we  now  have  well 
over  100  major  projects  that  are  using  this  very  parallel  machine. 
So  I  don't  know.  I  guess  maybe  it's  a  different  time. 

Dr.  ROSKIES.  I  concur  with  what  Mai  and  Larry  have  just  told 
you.  I  should  qualify  my  statement  that  I  just  made  that  we're  not 
going  to  cooperate  until  the  recompetition  is  over.  The  cooperation 
on  the  allocations  will  in  fact  continue. 

There  was  just  a  meeting  this  month  of  the  large,  of  this  commit- 
tee that  meets  once  a  year  to  allocate  time  on  the  machines,  and 
the  Pittsburgh/Illinois  Allocation  Committee  will  continue  to  meet 
quarterly. 

I  should  point  out,  maybe  it  isn't  clear,  but  there  is  no,  even 
though  the  Pittsburgh  supercomputing  center  is  operated  by  Carne- 
gie Mellon  and  the  University  of  Pittsburgh,  there  is  no  set  aside 
for  researchers  at  those  universities.  They  have  to  compete  on  the 
same  basis  for  time  as  researchers  from  any  other  university. 

Dr.  Pewitt.  Not  having  spent  my  whole  life  on  supercomputers 
but  in  other  areas,  the  allocation  process  for  limited  resources,  the 
scientific  community  has  learned  how  to  do  that. 

What  we've  heard  here  is  just  this  specific  instance. 

As  long  as  one  has  a  process  that  everyone  respects  where  alloca- 
tions are  made  on  the  basis  of  merit,  I  think  resources  are  est  used. 
It's  when  people  go  outside  that  process  that  we  get  misallocations, 
and  we  all  have  processes  in  place  to  do  that. 

There's  some  national  cooperation  and  then  there's  some  local  al- 
location but  it's  all  merit-based,  and  that's  the  only  way  it  should 
be  done,  in  our  view. 

Mr.  SCHIFF.  The  gentleman's  time  has  expired,  but  if  you  have 
an  additional  pressing  question,  it's  recognized. 

Mr.  Cramer,  recognized  for  five  minutes. 

Mr.  Cramer.  Thank  you.  I'll  try  to  be  brief  because  I  had  to  miss 
part  of  the  testimony  here. 

But  I'm  interested  in  hearing  more  about  your  users,  and  I  think 
when  I  left,  we  were  beginning  to  hear  Dr.  Smarr,  you  were  talking 
about  9,000  users. 

I'm  as  well  interested  in  collaborations  between  existing  state  re- 
gional high  performance  computing  centers,  if  those  exist. 

Would  any  of  you  or  all  of  you  comment  about  that,  if  you  would, 
your  users  and  collaborative  efforts? 

Dr.  Kalos.  Well,  all  of  the  centers  have  thousands  of  users  scat- 
tered all  around  the  country.  There's  no  special  geographic  correla- 
tion between  the  location  of  the  center  and  the  location  of  the  user. 

The  users  themselves,  the  actual  users,  when  we  allocate  re- 
sources, we  identify  the  research  with  the  principal  investigator, 
the  PI.  Most  of  the  actual  users  are  the  students,  graduate  stu- 
dents and  post-docs,  who  are  the  people  with  their  hands  on  the 
terminals  and  their  eyes  on  the  screens. 
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The  collaboration  with  centers  is  widespread.  We  have  partici- 
pated in  Metacenter  Regional  Alliances  with  North  Carolina  and 
some  of  the  other  centers.  Those  have  been  based  in  special 
projects  in  our  case  related  to  education  at  every  level. 

The  collaboration  on  scientific  research  has  not  been  so  strong, 
and  I  must  say  that  the  new  program  offers  the  possibility,  I  would 
say  the  anticipation,  that  we  will  have  tighter  collaboration  on  sci- 
entific developments,  on  software  developments,  on  development  of 
new  algorithms,  and  the  development  of  computing  environments 
that  are  much  more  productive. 

I  would  second  the  sentiments  that  there  is  a  great  deal  of  talent 
around  the  country  that  could  well  be  brought  into  the  program, 
and  we  expect  to  do  that. 

I  am  sure  all  of  the  centers  are  preparing  very  strong  proposals 
in  that  respect.  So,  while  I  do  deplore  the  disappearance  of  the 
Metacenter  kind  of  collaboration,  I  think  we  need  both. 

I  think  we  need,  as  Dr.  Young  said  earlier,  we  need  both  the  col- 
laboration of  a  tightly  knit  partnership,  and  the  collaboration 
among  the  partnerships  that  will  re-emerge  in  the  future. 

Mr.  Cramer.  Thank  you. 

Dr.  Smarr? 

Dr.  Smarr.  Yes.  I  am  not  sure  of  the  total  number,  although  I 
believe  it  may  well  be  in  the  materials  of  the  Hayes  Report.  I  know 
that  there  must  be  at  least  30,000  users  across  the  four  centers 
that  have  been  enabled. 

I  would  compare  that  to  the  time  when  I  was  running  my  pro- 
posal in  1983.  I  think  it  would  be  difficult  for  me  to  have  scared 
up  even  300  users  in  all  of  the  universities  in  the  country  using 
supercomputers . 

So  we  saw  a  100-fold  increase  that  happened  literally  almost 
within  three  or  four  years  of  the  opening  of  the  centers,  a  100-fold 
increase  of  the  best  minds,  particularly  young  minds,  throughout 
the  university  system,  several  hundred  universities. 

It  stimulated  each  of  the  campuses,  for  instance,  to  dig  up  their 
quadrangles  and  bury  fiber  optics  so  that  they  could  network  their 
campus  and  then  create  links  to  the  emerging  regional  centers,  all 
of  which  were  driven  by  the  ultimate  backbone  between  the 
supercomputer  centers. 

It  created  a  demand  pull  to  get  users  to  get  to  the  supercomputer 
centers.  I  think  it  is  inestimable  the  impact  that  the  centers  pro- 
gram has  had  on  the  everyday  working  environment  of  scientists 
and  researchers  throughout  hundreds  of  universities,  98  percent  of 
which  have  never  used  a  supercomputer  center  but  are  using  the 
infrastructure  that  only  came  into  being  because  of  the  genius  of 
the  National  Science  Foundation's  approach  to  this  program  of  set- 
ting up  the  centers  so  there  was  something,  some  pull,  and  I  think 
that  shows,  all  the  figures  show  that  the  NSF  networking  dollars, 
when  the  total  sum  was  added  up,  the  NSF's  federal  dollars  were 
a  few  percent  of  what  it  induced  to  be  spent  throughout  the  coun- 
try to  create  the  modern  world  of  the  Web. 

There  are  40  examples  of  real  breakthrough  science  that  were  ac- 
complished in  the  four  centers  that  are  in  the  appendix  to  the 
Hayes  Report  that  I  read  into  the  record.  Those  were  mutually 
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agreed  upon  across  the  four  centers  as  covering  all  the  possible  dis- 
ciplines. 

Mr.  Cramer.  Dr.  Smarr,  let  me  interrupt  you  here  because  I  am 
also  reviewing  my  notes.  Did  you  use  the  terms  "corporate 
partners"? 

Dr.  Smarr.  Yes. 

Mr.  Cramer.  Tell  me  a  little  about  that. 

Dr.  Smarr.  Surely. 

When  Eric  Bloch  particularly  was  Director  of  the  NSF,  there 
was — and  he  had  just  become  director  as  the  centers  were  an- 
nounced; I  think  it  was  one  of  his  first  press  conferences,  I  remem- 
ber— he  was  of  course  from  industry,  from  IBM,  and  he  was  very 
keen  that  the  federal  dollars  be  matched  by  dollars  added  to  the 
program  from  American  industry  on  a  voluntary  basis.  That  is,  the 
industry  had  to  decide  there  was  value  to  be  had  by  coming  into 
a  partnership 

Mr.  Cramer.  Like  a  subscriber? 

Dr.  Smarr.  Well,  actually 

Mr.  Cramer.  Or  a  user? 


Dr.  Smarr.  (continuing)  — what  is  really  interesting 

Mr.  Cramer.  Sort  of  like  a  pin  user? 

Dr.  Smarr.  (continuing)  — is  that  is  has  developed  from  the  tradi- 
tional academic  world  where  an  individual  in  a  corporation  knew 
somebody  who  was  a  professor,  maybe  had  been  a  student  there, 
and  would  give  $50,000  a  year  or  something  to  keep  some  research 
going,  in  which  case  I  would  say  that  is  not  strategic  partnering. 

What  you  have  now  is  strategic  partnering  between,  I'll  just 
mention  our  own  center  because  each  of  them  have  these  programs. 
We  have  maybe  16  large  corporate  partners.  We  set  out  to  choose 
them,  one  per  sector  of  the  economy,  so  that  we  would  be  able  to 
effect  the  entire  economy. 

Our  original  partners  are  places  like  Kodak,  and  Amoco,  and 
Dow  Chemical,  but  our  most  recent  partners  are  places  like  J.P. 
Morgan,  American  Airlines,  Sears,  and  Allstate. 

What  we  are  seeing  now  is  that  what  has  developed  here  in  high 
performance  computing  and  communication  is  much  more  broadly 
applicable  to  the  increase  in  competitiveness  of  our  corporate  econ- 
omy than  the  early  days  of  supercomputing. 

The  way  I  can  say  that  is,  originally  it  was  like  you  thought  you 
were  going  to  do  chemistry  or  structural  analysis,  so  that  would  be 
people  who  were  funded  by  the  R&D  division  of  the  corporation. 
That  is  maybe,  typically,  five  percent  of  the  annual  sales. 

What  about  the  other  95  percent  of  that  corporation? 

What  about  the  Congress? 

What  about  all  the  people  that  do  not  do  number  crunching  but 
do  vast  amounts  of  information  processing? 

That  is  where  this  emergence  of  the  microprocessor  technology 
such  as  IBM's  SP-2,  or  the  other  machines  that  are  around  at  the 
centers,  is  bringing  together  these  two  great  computing  worlds, 
that  of  information  processing  and  of  scientific  computing  into  one 
world. 

That  is  going  to  have  a  profound  impact  on  the  competitiveness 
of  the  economy.  I  believe  our  country  is  very  far  ahead  of  other 
countries  because  of  this  notion  that  from  the  beginning  NSF  had 
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that  we  should  go  out  and  encourage  industry  to  come  in  and  learn 
with  us  and  train  many  of  their  people. 

We  trained  150  bench  chemists  at  Eli  Lilly  to  become  computa- 
tional chemists,  as  well.  And  there  are  similar  stories  that  all  the 
centers  have. 

So  it  has  been,  I  think 

Mr.  Cramer.  Do  you  think  that  with  this  new  competition,  and 
perhaps  reducing  the  number  of  centers,  would  we  see  less  of  that 
and  it  would  be  more  concentrated?  Or  how  would  that 

Dr.  Smarr.  Well,  as  I  say 

Mr.  SCHIFF.  I  have  to  interrupt  just  because  time  is — I  would  in- 
vite a  brief  response,  if  you  would. 

Dr.  Smarr.  I  would  just  say,  again,  that  I  think  there  is  a  no- 
menclature error  that  is  being  made  over  and  over  again. 

While  it  is  true  that  the  number  of  leading  edge  centers  will  be 
reduced,  the  number  of  centers  or  research  groups  that  are  funded 
by  this  program  will  greatly  increase. 

Say  there  were  a  dozen  partners  per  two  centers  and  you  have 
24  centers  that  are  funded  instead  of  four. 

So  I  think  these  are  all  geographically  distributed.  Well,  so  is  in- 
dustry. So  I  think  that  the  chances  of  getting  more  interaction  with 
the  human  beings  in  industry  because  of  regional  centers  all  over 
the  country,  I  think  there  could  be  a  much  greater  impact  on  our 
economy  with  the  PACI  program  than  we  have  been  able  to  do  be- 
cause we  only  have  four  geographic  sites  from  which  to  do  it  in  the 
current  program. 

Mr.  Cramer.  Thank  you. 

Mr.  SCHIFF.  I  thank  the  gentleman. 

Mr.  Olver  is  recognized  for  five  minutes. 

Mr.  Olver.  Thank  you,  Mr.  Chairman. 

You  have  caught  me — it  is  absolutely  unthinkable  that  someone 
should  just  walk  in  and  be  actually  able  to  ask  questions  right 
away,  in  essence. 

[Laughter.] 

Mr.  Olver.  Let  me  ask  the  panelists,  since  we  are  talking  about 
supercomputing,  I  am  wondering.  Would  you  say  that  we  are  pre- 
eminent in  the  area  of  supercomputing  and  supercomputational  ca- 
pacities in  this  world? 

Dr.  Pewitt.  Yes. 

Dr.  Smarr.  Yes. 

Mr.  Olver.  We  are  preeminent.  The  definition  of  "preeminence" 
means  first  in  the  world. 

Dr.  Pewitt.  Yes. 

Dr.  Smarr.  I  think  there  is  no  question  that  we  are  first  in  the 
world,  but  if  you  go  back  to  ten  years  ago  and  you  compare  where 
we  were  relative  to  the  rest  of  the  world,  I  would  say  that  that  gap 
has  greatly  narrowed. 

Mr.  Olver.  Has  greatly  narrowed? 

Dr.  Smarr.  Yes.  Because  at  the  time  ten  years  ago,  we  were  the 
only  country  that  really  manufactured  advanced  supercomputers. 

Now,  if  you  look  at  the  top  500  fastest  computers  in  the  world, 
a  large  fraction  of  those  are  Japanese  built,  not  U.S.  built.  That 
would  not  have  been  the  case  10  years  ago.  There  would  have  been 
no  Japanese-built 
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Mr.  Olver.  a  large  fraction,  but  no  plurality? 

Dr.  Smarr.  Right. 

In  other  words,  the  United  States  still,  of  the  top  500  installed 
fastest  computers  in  the  world  from  all  vendors  all  over  the  world 
at  all  sites  all  over  the  world,  there  is  a  list  of  such  things  you  can 
go  look  at,  the  U.S.  still  dominates  the  production  of  the 
supercomputers  that  are  used  all  over  the  world,  but  there  are 
supercomputer  centers  modelled  on  the  NSF  centers  program  to  a 
large  extent  in  I  would  say  dozens  of  foreign  countries  now. 

But  I  do  not  believe  that  the — I  think  one  of  the  very  important 
things  that  this  country  did  in  the  last  five  years  was  the  high  per- 
formance computing  and  communications  program,  and  it  was  the 
coupling  of  the  "communications"  with  the  supercomputing  that 
has  really  given  our  country  the  advantage. 

Mr.  Olver.  Are  we  in  danger  of  losing  the  position  of  pre- 
eminence? 

[Pause.] 

Dr.  Kalos.  I  would  say 

Mr.  Olver.  Any  one  of  you? 

Dr.  Kalos.  I  would  say  there  is  a  possibility. 

First  of  all 

Mr.  Olver.  A  "possibility?"  Who  is  world  class  now  that  might 
endanger  our  preeminence  in  supercomputing? 

Dr.  Kalos.  First  of  all,  let  me  second  what  Larry  said.  When  it 
comes  to  the  integration  of  communications  and  computing  in  the 
scientific  life  of  our  country,  of  any  country,  we  do  lead  the  world. 

The  possible  competition  comes  from  Japan  where  the  funding  of 
large  computers  for  research  in  universities  is  very  strong.  I  know 
of  some  cases  in  Japan  where  large-scale  supercomputers,  several 
large,  state  of  the  art  supercomputers,  are  acquired  for  a  single 
university. 

So  we  are  not  talking  about,  in  Japan,  three  or  four  leading  cen- 
ters for  the  country  as  a  whole;  we  are  talking  about  some  dozens. 

Mr.  Olver.  Are  there  other  national  programs  that  are  at  the 
level  that  could  endanger  our  preeminence? 

Dr.  Kalos.  No. 

Mr.  Olver.  Only  Japan? 

Dr.  Kalos.  Only  Japan's. 

Mr.  Olver.  Who  else  would  you  say  is  world  class? 

Dr.  Smarr.  It  depends  on  what  you  mean,  whether  you  are  talk- 
ing about  the  use  of  the  supercomputers,  or  the  manufacturing  of 
the  supercomputers.  Those  are  two  very  different  worlds. 

Mr.  Olver.  Well  I  suppose  I  am  thinking  of  more  even  than  that. 
The  research  that  creates  the  supercomputers'  manufacturing  fol- 
lowing the  development  of  and  the  advancements  that  need  to  be 
made  or  may  need  to  be  made  to  maintain  preeminence  as  far  as 
the  production  and  manufacturing,  and  in  the  use.  Those  are — you 
have  to  have  the  first  before  you  can  have  the  second  and  then  the 
third. 

Dr.  Kalos.  No,  you  can  buy  the  supercomputers  from  other  coun- 
tries. 

However,  there  is  only  one  rival  in  both  the  basic  research  that 
underlies  all  computing  and  communication  technologies,  the  devel- 
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opment  and  the  production  of  high-end  computers,  and  that  is 
Japan. 

As  far  as 

Mr.  Olver.  That  is  only  one  rival. 

Dr.  Kalos.  There  is  only  one  rival. 

As  far  as  use,  there  are  strong  programs  around  the  world.  There 
is  a  very  strong  program  in  Japan,  as  I  mentioned.  There  are  excel- 
lent scientists  in  every  country  I  have  visited — England,  France, 
Sweden,  Israel. 

Mr.  Olver.  Has  somebody  asked  the  question.  What  must  we  do 
to  maintain  the  preeminence? 

I  assume 

Dr.  Smarr.  Not  here  today. 

Dr.  Kalos.  No.  No  one  has  asked  that  question. 

Mr.  Olver.  Can  you  give  some  short — because  I  know  the  chair- 
man will — probably  the  red  light  has  already  gone  off. 

Mr.  SCHIFF.  Your  sense  of  timing  is  excellent,  but  I  will  accept 
a  brief  answer. 

[Laughter.] 

Mr.  Olver.  If  we  could  get  a  brief  answer  of  not  more  than  30 
seconds  to  a  minute  from  each  of  you,  if  you  would  like. 

Dr.  Kalos.  I  would  say,  ensure  that  we  have  a  strong  competi- 
tive NSF  program  along  the  lines  that  have  been  outlined  today. 
Make  sure  it  is  well  funded  over  the  next  ten  years.  Please  pay  at- 
tention to  it  not  just  now  but  two  years  from  now,  five  years  from 
now,  ten  years  from  now. 

Dr.  Smarr.  I  think  the  program  that  we  are  testifying  on  today 
from  the  NSF,  the  PACI  program,  the  Partnerships  for  Advanced 
Computational  Infrastructure,  if  the  budget  levels  are  maintained 
over  the  anticipated  five  to  ten-year  period,  will  guarantee  that  this 
country  stays  preeminent  if  the  solicitation  is  carried  out  with 
merit  review  and  not  with  any  political  involvement. 

Mr.  Olver.  If  the  current  funding  is  maintained? 

If  the  current  funding  is  maintained,  we  will  maintain  our  pre- 
eminence? 

Dr.  Smarr.  I  think  if  we  can  maintain  it  in  real  terms.  The  prob- 
lem we  get  into  is  if  we  are  talking  about  keeping  it  in  the  same 
current  dollars.  Then,  over  five  years  most  projections  are  a  25  per- 
cent reduction  in  real  terms. 

If  we  are  talking  real  terms,  that  it  stays  in  the  same  level — that 
is,  inflationary  increases — I  think  the  program  can  do  it. 

Dr.  ROSKIES.  I  agree  with  what  my  colleagues  said.  I  would  like 
to  add  just  one  more  point.  That  is,  there  are  other  agencies  in  the 
government  that  are  putting  significant  resources  into  high-per- 
formance computing,  but  I  think  if  you  want  to  retain  preeminence 
for  the  country  as  a  whole,  the  NSF  program  that  reaches  out  to 
all  universities  in  the  training  of  students  and  so  on  is  absolutely 
critical. 

Dr.  Pewitt.  It  is  a  competitive  world.  They  want  what  we  have 
got.  Any  slackening  of  what  we  are  doing  just  invites  loss  of  what 
we  have. 

It  is  that  simple. 

Mr.  Olver.  Thank  you,  Mr.  Chairman. 
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Mr.  SCHIFF.  If  there  are  no  other  Members  seeking  recognition, 
let  me  thank  this  panel. 

Before  you  are  excused,  let  me  just  reiterate  what  I  said  at  the 
beginning  of  the  hearing.  Please  do  not  be  concerned  about  our 
coming  and  going.  I  know  it  is  a  little  disconcerting.  It  is  not  in- 
tended to  be  disrespect  towards  you  or  any  of  the  other  witnesses; 
it  is  the  necessity  of  having  to  cover  several  things  at  once  each 
day. 

We  can  assure  you  that  the  record  remains  whole,  and  that  is 
the  most  important  purpose  of  the  hearing,  in  my  judgment. 

With  that,  I  would  like  to  invite  the  third  panel  here  to  come  up, 
please. 

The  third  panel  is  Dr.  Mary  Vernon  of  the  Department  of  Com- 
puter Science  and  Engineering  at  the  University  of  Washington, 
Seattle,  Washington;  Dr.  Douglas  Gale,  Assistant  Vice  President  of 
Information  Systems  and  Services  of  the  George  Washington  Uni- 
versity in  Washington,  D.C.;  and  Dr.  Kelvin  Droegemeir,  CAPS  Di- 
rector, University  of  Oklahoma,  Norman,  Oklahoma. 

Let  me  just  ask  first.  I  am  pretty  comfortable  that  I  pronounced 
the  last  names  "Vernon"  and  "Gale"  correctly.  Dr.  Droegemeir,  did 
I  do  well  on  yours?  Or  have  I  mispronounced  your  name? 

Dr.  Droegemeir.  Kelvin. 

Mr.  SCHIFF.  Kelvin  is  your  first  name,  but  Doctor 

Dr.  Droegemeir.  Kelvin  "Dro-gah-myer". 

Mr.  SCHIFF.  Droegemeir  is  the  right  pronunciation? 

Dr.  Droegemeir.  Yes. 

Mr.  SCHIFF.  I  appreciate  it. 

Welcome.  Dr.  Vernon,  let  me  say  that  yours  and  the  other  mem- 
bers of  the  panel's  written  statements  will  be  made  part  of  the 
record.  You  are  welcome  to  summarize  as  you  see  fit. 

STATEMENT  OF  DR.  MARY  VERNON,  DEPARTMENT  OF  COM- 
PUTER SCIENCES  AND  ENGINEERING,  UNIVERSITY  OF 
WASHINGTON,  SEATTLE,  WASHINGTON 

Dr.  Vernon.  Thank  you,  Mr.  Chairman,  and  Members  of  the 
Subcommittee: 

Thank  you  for  the  opportunity  to  testify  on  the  National  Science 
Foundation  Program,  Partnerships  for  Advanced  Computational  In- 
frastructure. 

My  name  is  Mary  Vernon,  and  I  am  a  Professor  of  Computer 
Science  with  a  joint  appointment  in  Industrial  Engineering  at  the 
University  of  Wisconsin  at  Madison.  I  am  currently  on  sabbatical 
at  the  University  of  Washington  in  the  Computer  Science  and  En- 
gineering Department. 

I  am  the  project  director  of  the  Computer  Science  Department's 
Laboratory  for  Research  and  Next  Generation  Parallel  Systems  at 
the  University  of  Wisconsin. 

My  research  involves  developing  models  to  estimate  the  perform- 
ance impact  of  proposed  design  changes  for  high-performance  com- 
puters. So  in  other  words,  my  research  is  on  the  underljdng  tech- 
nologies for  high-performance  computing. 

I  also  am  a  member  of  the  Board  of  Directors  of  the  Computing 
Research  Association,  a  scientific  society  whose  membership  con- 
sists of  nearly  all  the  graduate  departments  of  computer  science 
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and  computer  engineering  in  the  United  States,  and  affiliated  pro- 
fessional societies,  as  well  as  several  U.S.  industrial  research  lab- 
oratories that  perform  basic  research  in  computing. 

My  knowledge  of  the  supercomputer  centers  comes  from  collabo- 
rations related  to  my  research,  as  well  as  fi-om  my  participation  in 
the  two  recent  panels  that  evaluated  the  program,  the  1993  NSF 
Blue  Ribbon  Panel  on  High  Performance  Computing  that  was 
chaired  by  Louis  Branscomb,  and  the  1995  NSF  Task  Force  chaired 
by  Edward  Hayes. 

Today  I  would  like  to  explain  why  I  strongly  endorse  both  the  re- 
structuring of  the  program  as  proposed  in  the  Task  Force  Report 
and  the  planned  recompetition  process  as  described  in  the  NSF 
Program  Solicitation. 

I  would  like  to  make  six  key  points: 

The  first  point  is  that  the  accomplishments  of  the  NSF 
Supercomputing  Centers  over  the  past  ten  years  have  been  tremen- 
dous. You  have  heard  a  great  deal  of  testimony  about  this,  I  am 
sure  in  the  past,  as  well  as  today. 

But  I  would  like  to  also  reiterate  that  the  Centers  have  provided 
competing  capabilities  that  are  significantly  beyond  what  is  pos- 
sible in  individual  research  universities. 

At  Wisconsin  in  the  Laboratory  for  research  on  Next-Generation 
Parallel  Systems,  we  have  a  64-node  CM-5  system.  This  is  one  of 
those  mid-range  systems  that  you  have  been  hearing  about  that 
allow  us  to  do  some  experimentation  at  a  level  below  what  the  na- 
tional centers  can  provide. 

It  is  a  very  useful  system  for  code  development  and  testing,  and 
also  for  small-scale  experimentation  with  new  algorithms  and  sys- 
tem design  issues. 

However,  we  rely  on  the  512-node  system,  512-node  CM-5  sys- 
tem, at  the  National  Center  for  Supercomputing  Applications  for 
our  larger  scale  experimentation. 

Similarly,  we  have  recently  acquired  an  8-node  IBM  SP-2  system, 
and  we  will  be  relying  on  the  512-node  system  at  the  Cornell  The- 
ory Center  again  for  our  larger  scale  experimentation. 

The  Center's  capabilities  have  enabled  fundamental  advances  in 
science  and  engineering  as  documented  in  Appendix  B  of  the  Task 
Force  Report.  I  was  unaware  of  the  extent  of  the  accomplishments 
in  other  fields  prior  to  my  participation  in  the  Branscomb  Panel, 
but  I  quickly  learned  that  the  program  is  serving  the  Nation  and 
the  broader  science  and  engineering  community  extremely  well. 

The  vision  that  created  the  program  was  excellent.  Furthermore, 
a  survey  of  the  science  and  engineering  research  community  in  Ap- 
pendix G  of  the  Task  Force  Report  shows  that  for  large  numbers 
of  researchers  m  all  disciplines  the  centers  have  been  indispen- 
sable. 

The  second  point  that  I  would  like  to  make  is  that  the  NSF  Cen- 
ters Program  has  been  thoroughly  studied  by  a  broad  spectrum  of 
science  and  engineering  researchers.  Both  have  had  productive 
uses  for  the  funding. 

There  is  widespread  agreement  concerning  the  need  for  a  future 
program  as  recommended  in  the  Task  Force  Report. 

The  scientific  goals  of  the  Centers  Program  are  paramount.  The 
1993  NSF  Blue  Ribbon  Panel  chaired  by  Louis  Branscomb  had  rep- 
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resentation  from  applied  mathematics,  computer  science,  physics, 
chemistry,  biology,  and  civil,  chemical,  and  environmental  engi- 
neering. 

The  1995  Task  Force  had  similarly  broad  representation,  as  you 
have  heard  earlier  today.  Both  reports  came  to  a  similar  conclusion 
about  the  success  of  the  Centers  Program  over  the  past  10  years, 
and  the  importance  of  continuing  to  provide  access  to  leading  edge 
capabilities  for  scientists  and  engineers  in  the  next  decade. 

The  third  point  that  I  would  like  to  make  is  that  the  program 
will  be  significantly  strengthened  by  the  proposed  restructuring  to 
include  leading  edge  sites  that  are  strongly  partnered  with  experi- 
mental facilities  and  research  centers  at  other  universities,  as  well 
as  other  national  and  regional  high-performance  computing  cen- 
ters. 

In  the  past,  leading  edge  centers  have  learned  about  new  high- 
performance  computing  technology  primarily  from  vendors  and  in- 
formal relationships  with  NSF  research  centers. 

Formal  partnering  with  the  experimental  facilities  and  research 
centers  at  other  universities  will  expand  the  center  capabilities  for 
earlier  and  broader  experimentation  with  new  high-performance 
technologies. 

For  example,  there  are  a  number  of  computer  science  groups  at 
the  University  of  California  at  Berkeley,  at  Wisconsin,  at  the  Uni- 
versity of  Washington,  and  other  places  that  are  experimenting 
with  networks  of  workstation  technologies  that  could  eventually 
support  production  high  performance  computing  applications. 

Partnerships  with  these  groups  can  enable  NSF's  Leading  Edge 
Centers  to  experiment  with,  anticipate,  and  prepare  for  next  gen- 
eration technologies  more  thoroughly  than  has  previously  been  pos- 
sible. 

For  the  same  reason,  formal  partnerships  with  research  groups 
developing  next-general  computational  algorithms  and  techniques 
will  also  be  important. 

You  have  already  heard  today  that  other  partnerships  that  in- 
clude coordinated  planning  of  resource  allocation  with  university, 
regional,  or  national  high-performance  computing  centers  will  en- 
able more  cost-effective  use  of  the  Nation's  computing  infrastruc- 
ture. 

The  fourth  point  that  I  would  like  to  make  is  that  the  require- 
ment that  leading  edge  centers  provide  balanced  computing  capa- 
bilities that  are  one  or  two  orders  of  magnitude  beyond  what  is 
available  at  leading  research  universities,  together  with  recent 
trends  in  technology  and  the  computer  industry,  make  it  both  pos- 
sible and  necessary  to  reduce  the  number  of  leading  edge  sites. 

I  would  like  to  explain  this  as  carefully  as  I  can. 

In  the  past.  Vector  supercomputers  were  a  reasonably  stable 
technology.  A  single  system  could  be  considered  leading  edge  in 
terms  of  both  computing  power  and  computing  technology  over  a 
period  of  say  five  years. 

But  due  to  the  introduction  of  parallel  supercomputers  which 
have  a  much  greater  pace  of  evolution,  leading  edge  systems 
change  significantly  now  on  a  three-year  time  frame  or  there- 
abouts. You  have  heard  two-and-a-half  years,  or  18  months,  but  I 


191 

will  pick  three  years  as  the  number  that  we  feel  doing  research  at 
Wisconsin  and  Washington. 

The  current  centers  program  has  required  an  influx  of  funding 
from  other  sources  in  order  to  keep  up  with  recent  advances  in 
high-performance  systems.  Even  so,  centers  are  financially  limited 
in  how  quickly  they  can  purchase  next-generation  systems. 

Science  and  engineering  advances  would  benefit  if  a  more  rapid 
upgrade  path  at  the  leading  edge  sites  could  be  achieved. 

My  next  point  is  particularly  important: 

In  addition  to  keeping  pace  with  advancing  high-  performance 
computer  technology,  it  is  important  that  the  leading  edge  centers 
provide  balanced  systems. 

As  processors  become  faster,  this  means  purchasing  larger 
amounts  of  memory  and  more  expensive  disk  subsystems  for  each 
machine. 

Furthermore,  there  are  significant  advantages  to  aggregating 
both  computing  power  and  memory  at  fewer  sites  as  this  enables 
the  solution  of  even  larger  and  more  complex  system  models. 

The  aggregation  of  computing  power  and  memory  is  becoming 
more  important  for  leading  edge  computational  science  and  engi- 
neering than  the  ability  to  access  machines  from  different  vendors. 

Thus,  the  trends  in  computer  architecture  and  the  need  for  larg- 
er scale  balanced  systems,  each  argue  compellingly  for  a  smaller 
number  of  centers,  each  funded  at  a  higher  level. 

This  point  is  well  understood  by  NSF  and  the  computational 
science  and  engineering  community,  but  is  perhaps  not  necessarily 
explained  in  the  Task  Force  Report. 

My  fifth  point  is  that  recompeting  the  centers  is  the  best  ap- 
proach, I  believe,  to  achieving  the  most  effective  leading  edge  sites 
and  partnerships. 

As  the  Task  Force  began  its  work,  we  were  all  well  aware  of  the 
concern  expressed  in  the  1993  Blue  Ribbon  Panel  Report  about  the 
potential  risks  of  recompetition  of  the  existing  centers. 

Two  of  the  Task  Force  members,  myself  included,  had  served  on 
the  panel  chaired  by  Louis  Branscomb.  Thus,  the  Task  Force  as  a 
whole,  and  I  in  particular,  started  out  with  a  healthy  level  of  con- 
cern about  the  risks  associated  with  recompetition. 

There  are  strong  arguments  against  recompeting  the  centers. 
Most  importantly,  invaluable  expertise  at  the  existing  centers  and 
the  need  for  stability  in  order  to  maintain  that  expertise. 

At  least  one  former  center  staff  member  commented  in  response 
to  the  survey  in  Appendix  G  of  the  Task  Force  Report  that,  "A  pri- 
mary reason  for  my  leaving  the  centers  program  was  an  unease 
about  NSF's  commitment  to  the  program." 

However,  from  1993  to  1995,  we  noted  several  key  developments 
that  changed  the  balance  in  favor  of  new  competition. 

In  addition  to  the  usual  advantages  of  recompetition  such  as  the 
degree  of  creativity  and  renewed  commitment  that  a  program  solic- 
itation invites,  the  Task  Force  concluded  that  there  had  been  im- 
portant changes  in  the  technology  that  called  for  a  new  structure 
for  the  centers,  a  structure  based  on  partnerships  between  leading 
edge  sites  and  other  partnership  centers  with  significant  technical 
expertise,  as  you  have  heard  about  earlier. 
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As  a  result,  the  Task  Force  concluded  that  a  whole  new  competi- 
tion was  needed.  Such  a  change  is  best  accomplished  by  an  open 
competition  and  merit  review. 

In  the  end,  the  likely  increased  overall  strength  of  the  responses 
to  the  competitive  solicitation  outweighs  the  significant  stability  is- 
sues. 

My  final  point  is  that  members  of  the  broad  scientific  and  engi- 
neering community  following  the  merit  review  process  and  criteria 
outlined  in  the  program  solicitation  will  make  wise  choices  for  the 
Nation. 

It  is  likely  that  such  a  broad  spectrum  of  expertise  has  never 
been  brought  to  bear  on  a  decision.  The  review  panel  members  will 
fully  understand  the  intricacies  of  the  new  directions  and  will  best 
be  able  to  judge  the  number  and  combination  of  proposed  partner- 
ship sites  that  will  best  serve  computational  science  and  engineer- 
ing and  the  national  interest  in  the  next  decade. 

In  summary,  Mr.  Chairman,  I  strongly  support  the  restructuring 
of  the  NSF  Supercomputers  Program  to  Create  Partnerships  for 
Advanced  Computational  Infrastructure.  These  will  provide  leading 
edge  computational  facilities  as  well  as  coordinated  resource  alloca- 
tion and  experimentation  with  new  technologies  on  mid-range  sys- 
tems. 

The  partnerships  should  be  funded  at  a  level  that  allows  the 
leading  edge  systems  to  keep  pace  with  changes  in  high  perform- 
ance computing  technologies  and  to  provide  balanced  processor 
memory  and  input/output  capabilities  that  are  one  or  two  orders  of 
magnitude  beyond  what  will  be  available  at  major  research  univer- 
sities in  the  near  future. 

The  planned  NSF  review  process  is  the  appropriate  mechanism 
for  determining  the  specific  number  and  types  of  partnerships  that 
will  best  meet  the  needs  of  computational  science  and  engineering 
in  the  next  decade. 

Thank  you  again  for  the  opportunity  to  testify. 

Mr.  SCHIFF.  Thank  you.  Dr.  Vernon. 

Dr.  Droegemeier,  you  are  recognized. 

STATEMENT  OF  DR.  KELVIN  K.  DROEGEMEIER,  SCHOOL  OF 
METEOROLOGY  AND  CENTER  FOR  ANALYSIS  AND  PRE- 
DICTION OF  STORMS,  UNIVERSITY  OF  OKLAHOMA 

Dr.  Droegemeier.  Thank  you,  Mr.  Chairman,  and  good  after- 
noon, and  Members  of  the  Committee: 

I  would  like  to  thank  you  for  the  privilege  of  appearing  before 
you  to  speak  this  afternoon.  My  name  is  Kelvin  Droegemeier.  I  am 
an  Associate  Professor  of  Meteorology  at  the  University  of  Okla- 
homa, and  I  am  Director  of  one  of  NSF's  25  Science  and  Technology 
Centers,  the  Center  for  Analysis  and  Prediction  of  Storms,  also  at 
the  University  of  Oklahoma. 

I  would  like  to  share  my  thoughts  concerning  the  NSF 
Supercomputer  Centers  and  recompetition  by  describing  a  particu- 
lar four-year  partnership  between  my  center  and  the  Pittsburgh 
Supercomputing  Center. 

This  project  is  poised  to  help  redefine  the  concept  of  weather  pre- 
diction, and  in  many  ways  is  a  living,  breathing  example  of  the 
t5T)es  of  partnerships  we  have  been  talking  about  this  afternoon. 
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The  goal  of  my  center  is  to  develop  techniques  for  and  operation- 
ally evaluate  numerical  prediction  of  individual  thunderstorms  and 
wintertime  storms  up  to  six  hours  in  advance,  as  opposed  to  for  ex- 
ample like  today  outside  there  is  maybe  a  60  percent  chance  of 
rain.  The  goal  of  my  Center  is  to  be  able  to  tell  you  specifically  at 
what  time  during  the  day  such  events  will  occur  up  to  six  hours 
in  advance. 

The  potential  economic  impact  of  this  capability  is  staggering,  as 
private-sector  and  government  interests  lose  well  over  a  billion  dol- 
lars ever  year  as  a  result  of  weather  events  that  occur  on  scales 
much  smaller  than  could  be  predicted  using  current  technology. 

For  example,  in  the  case  of  the  Oklahoma  City  bombing,  the  re- 
covery efforts  could  have  been  aided  substantially  by  short-term  ac- 
curate, say  on  the  order  of  six  hour,  forecasts  of  things  like  precipi- 
tation, winds,  and  lightening. 

The  ability  of  my  science  and  technology  center  to  accomplish  its 
mission  depends  critically  upon  access  to  and  the  effective  use  of 
high-performance  computing  and  telecommunications  systems. 

In  most  areas  of  science,  getting  results  quickly  is  a  desirable — 
is  very  desirable,  but  in  meteorology  it  is  absolutely  essential.  If 
you  can't  predict  the  weather  faster  than  it  occurs,  the  prediction 
has  absolutely  no  value  at  all. 

This  poses  an  extreme  computational  challenge  and  is  uniquely 
suited  for  exploration  at  the  NSF  National  Centers. 

During  the  past  four  years,  my  center  has  been  conducting  a  se- 
ries of  real-time  operational  tests  of  this  new  storm  scale  prediction 
capability  over  the  southern  Great  Planes.  These  tests  require  ab- 
solutely dedicated  use.  That  is,  dedicated  entirely  to  our  project,  of 
Pittsburgh's  two  largest  machines  for  four  to  six  hours  each  day 
during  a  two-month  operational  period  during  which  these  tests 
were  being  conducted. 

This  allocation  of  computer  time  and  resources  was  based  on 
merit  review,  as  you  have  heard  from  the  previous  panel.  To  facili- 
tate this  complex  effort,  Pittsburgh  also  dedicated  two  technical 
support  staff  to  us — and  I  mean  "dedicated" — for  the  entire  two 
months  of  our  project,  as  well  as  other  personnel  who  served  in  a 
consulting  role. 

By  working  closely  with  the  Pittsburgh  center  in  this  manner, 
my  center  was  able  to  prototype  various  computational  strategies 
and  techniques  for  operational  storm  scale  prediction  that  can  be 
considered  by  the  National  Weather  Service  for  implementation 
early  in  the  next  century. 

In  fact,  the  results  of  our  1995  Spring  Operational  Tests  were  so 
encouraging  that  a  major  U.S.  commercial  airline,  in  fact  the  larg- 
est airline  in  the  world,  is  now  in  the  final  stages  of  negotiating 
a  three-year,  $1  million  initial  research  and  development  contract 
with  my  science  and  technology  center. 

This  private  sector  involvement,  which  brings  additional  private- 
sector  support  to  two  federally  funded  centers — that  is,  my  science 
and  technology  center  funded  by  the  NSF,  and  Ralph  Roskies  Pitts- 
burgh Supercomputing  Center,  and  which  seeks  also  to  customize 
this  particular  weather  prediction  technology  for  commercial  avia- 
tion use  would  absolutely  have  not  have  been  possible  without  the 
dedicated  supercomputing  time,  but  also  more  importantly  the  very 
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extensive  collaboration  scientifically  that  was  offered  by  the  Pitts- 
burgh Center. 

Similar  customizations  of  our  technology  are  now  being  pursued 
in  other  areas  of  the  private  sector  and  the  government  as  well,  in- 
cluding the  United  States  Air  Force. 

The  restructuring  of  the  Supercomputers  Centers  Program  is 
taking  place,  as  you  have  heard,  in  an  environment  significantly 
different  than  existed  five  years  ago  or  ten  years  ago  rather  when 
the  original  five  centers  were  established. 

Most  notable,  as  we  have  been  hearing,  is  the  proliferation  of  in- 
expensive, mid-range  computing  power  of  significant  capacity  in 
scientists'  laboratories  and  on  their  desktops. 

Those  projects  now  that  are  capable  of  being  uniquely  performed 
at  the  national  centers  are  of  the  type  that  require  specialized  and 
dedicated  resources,  and  especially  close  technical  and  scientific 
support  for  extended  periods  of  time. 

No  longer  are  we  in  the  concept  of  providing  general  cycles  to 
thousands  of  users.  It  is  a  much  more  specialized  arena  that  we  are 
talking  about  getting  into. 

In  this  regard,  I  believe  the  concept  for  the  recompetition  is  a 
very  good  one.  I  think  care,  though,  must  be  taken  in  evolving  this 
new  infrastructure  to  avoid  two  potential  problems. 

The  first  is  a  reduction  in  overall  resources  available  to  the  na- 
tional community. 

The  second  is  an  elimination  potentially  of  diversity  that  exists 
among  the  current  centers. 

I  believe  that  if  the  current  four  principal  centers  are  consoli- 
dated into  one  or  two,  even  with  the  appropriate  levels  of  funding 
they  might  well  lose  their  effectiveness  in  collaborating  with  their 
users.  This  is  certainly  not  a  desirable  end  in  this  recompetition. 

Therefore,  I  believe  any  reduction  in  overall  funding  to  the  cen- 
ters would  be  unproductive  and  unwise  and  it  could  greatly  dimin- 
ish our  ability  to  move  science  and  technology  from  the  university 
sector  into  other  areas  that  benefit  society,  especially  the  private 
sector. 

I  think  it  is  also  important  to  recognize  and  preserve  the  diver- 
sity and  the  individuality  that  now  exists  among  the 
supercomputing  centers  that  really  is  an  underpinning  for  the 
strength  of  the  entire  program. 

The  new  Centers,  like  the  old,  should  be  provided  with  sufficient 
latitude  to  establish  our  own  identities.  Each  center  cannot  and 
should  not  be  all  things  to  all  people.  If  that  had  been  the  case 
with  the  Pittsburgh  Supercomputing  Center,  the  efforts  we  have 
now  been  successful  with  in  my  view  would  not  have  happened  at 
all. 

Thank  you,  very  much. 

[The  prepared  statement  of  Dr.  Droegemeier  follows:] 
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I  wish  to  offer  a  somewhat  unique,  4-way  perspective  of  the  NSF  supercomputing 
centers  that  I  have  gained  during  the  past  13  or  so  years.  My  first  perspective  was 
established  as  a  graduate  student  at  tne  University  of  IlUnois  in  the  early  1980's. 
At  that  time,  one  of  the  principal  visionaries  for  a  national  centers  program.  Dr. 
Larry  Smarr,  was  working  closely  with  my  doctoral  advisor.  I  was  fortunate  to 
watch  at  close  range  the  unfolding  of  the  proposal  to  create  the  Illinois  center 
(NCSA)  and  to  participate  in  site  visits  and  other  preparatory  activities.  The  NCSA 
was  being  installed  just  as  I  graduated.  My  second  perspective  comes  as  a  user  of 
supercomputers  since  1980  and  the  national  centers  since  their  inception,  and  as  a 
professor  who  continues  to  guide  students  in  the  use  of  these  resources.  Thirdly,  I 
served  for  4  years  on  the  joint  NCSA/PSC  Peer  Review  Board,  which  on  a  quarterly 
basis  allocates  resources  at  the  Illinois  and  Pittsburgh  centers  based  on  written  pro- 
posals. This  service  afforded  me  an  inside  view  of  the  topics  being  investigated  with 
supercomputers,  the  division  of  computing  resources  among  disciplines,  and  the  role 
of  these  two  centers  in  shaping  the  computing  literacy  and  scientific  productivity  of 
the  national  research  enterprise.  Finally,  for  the  past  4  years  I  have  been  working 
on  a  special  joint  project  between  the  Pittsburgh  Supercomputing  Center  (PSC)  and 
the  NSF  Science  and  Technology  Center  (STC)  which  I  direct.  This  project  seeks  to 
assemble  and  evaluate  the  computational  infrastructure  for  a  prototype  system  de- 
signed to  numerically  predict  the  occurrence  of  individual  severe  spring  and  winter- 
time storms  up  to  6  hours  in  advance.  If  successful  and  implemented  by  the  Federal 
or  private  sectors  or  both,  this  effort,  which  would  have  been  impossible  without  the 
NSF  supercomputing  centers  program,  could  yield  millions  of  dollars  in  savings  an- 
nually across  all  sectors  of  the  U.S.  economy. 

With  that  as  a  preface,  I  would  like  to  share  my  personal  experiences  regarding 
the  NSF  supercomputing  centers  program,  my  thoughts  on  the  proposed  restructur- 
ing, and  suggestions  regarding  the  re-competition  process. 

I  am  an  associate  professor  of  meteorology  at  the  University  of  Oklahoma  and  di- 
rector of  one  of  the  NSF's  25  Science  and  Technology  Centers — the  Center  for  Analy- 
sis and  Prediction  of  Storms  (CAPS).  Since  the  late  1940's,  when  computers  emerged 
as  potentially  powerful  tools  and  were  first  applied  to  predicting  the  global  weather, 
the  science  of  meteorology  has  been  a  strong  driving  force  behind  their  continued 
evolution.  In  most  areas  of  basic  research,  getting  results  quickly  is  desirable.  In 
meteorology,  it  is  essential:  if  one  cannot  predict  the  weather  significantly  faster 
than  it  evolves,  the  prediction  has  no  value.  Scientists  and  students  in  my  discipline 
have  enjoyed  a  long  history  of  access  to  very  powerful  computers  at  the  National 
Center  for  Atmospheric  Research.  Most  other  disciplines  in  the  U.S.  were  not  so  for- 
tunate, however,  as  high-performance  computing  systems  were  generally  confined  to 
DoE  or  other  government  laboratories  and  were  inaccessible  to  the  general  scientific 
and  engineering  communities. 

Recognizing  the  need  for  such  resources  and  the  associated  scientific,  economic, 
and  societal  benefits  to  the  nation,  the  NSF  national  centers  program  was  estab- 
lished. It  is  important  to  note  that  these  centers  were  located  exclusively  at  high- 
quality  research  universities,  in  some  cases  with  strong  linkages  to  private  industry. 
The  importance  of  this  structure  cannot  be  overstated;  it  allowed  the  academic  en- 
terprise in  general,  and  basic  research  in  particular,  to  guide  the  evolution  and 
modus  operandi  of  the  centers.  In  effect,  it  was  recognized  that  those  needing  the 
resources  were  uniquely  positioned  to  best  determine  how  they  should  be  provided, 
and  in  large  part  this  explains  the  success  of  the  program  today. 

The  numerous  accomplishments  of  the  NSF  centers  have  been  documented  exten- 
sively and  advertised  effectively.  From  my  personal  perspective,  however,  the  cen- 
ters have  provided  not  only  high  performance  computing  cycles  to  the  nation's  sci- 
entists, but  also  have  helped  lead  a  revolution  in  the  approach  taken  to  solve  chal- 
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lenging  problems.  Scientists  now  tend  to  view  their  research  not  in  isolation,  at- 
tempting to  solve  individual  problems  in  a  particular  area  of  study,  but  as  compo- 
nents of  coupled  systems  where  important  interdependencies  are  no  longer  ne- 
glected. Meteorologists  now  work  closely  with  biologists  and  hydrologists,  physicists 
with  cognitive  scientists.  In  essence,  the  centers  re-invigorated  the  research  enter- 

Erise  by  fostering  communication  among  traditionally  separate  disciplines,  and  have 
ecame  a  focal  point  for  multi-disciplinary  interaction — long  before  the  phrase  be- 
came fashionable.  Moreover,  the  centers  established  computers  as  legitimate  tools 
in  the  research  process — as  tools  of  basic  scientific  discovery  rather  than  the  "easy 
way  out"  if  analytic  theories  proved  difficult. 

In  addition  to  fulfilling  their  basic  mission,  the  NSF  national  centers  have  had 
other  significant  positive  effects  on  science  and  education.  They  fostered  the  genera- 
tion of  tools  designed  to  assist  scientists  with  their  research — visualization,  code 
analysis  and  optimization,  and  performance  tools,  and  tools  whereby  a  scientist  can 
"step  into"  a  numerical  calculation  as  it  is  being  performed  and  dynamically  adjust 
parameters  (e.g.,  putting  out  the  flaps  of  an  aircraft  wing  as  the  angle  of  attack  is 
changing).  Students  in  grades  K-12  have  received  first-hand  experience  with  high- 
performance  computing  resources  through  special  educational  programs  at  the  na- 
tional centers,  and  now  some  are  using  computers  in  the  same  manner  as  did  grad- 
uate students  only  a  few  years  ago.  Through  their  "industrial  partners"  programs, 
the  centers  were  perhaps  the  first  to  fuse  private  sector  and  university  researchers 
in  a  meaningful  way,  setting  the  stage  for  the  new  paradigm  of  private  sector  in- 
volvement in  academic  research  and  education  that  has  been  further  solidified  by 
the  NSF  STC  program.  Finally,  the  centers,  much  like  the  space  program  in  the 
1960's  and  1970's,  brought  collateral  benefits  to  society — the  Internet,  World  Wide 
Web,  and  other  innovations  in  information  technology.  In  large  part  these  resources 
arose  by  virtue  of  the  need  for  our  nation's  scientists  and  engineers  to  gain 
highspeed  access  to  the  NSF  supercomputing  centers. 

A  perhaps  elusive  fact  is  that  the  national  centers  established  a  credible  conduit 
between  scientists  and  computer  manufacturers  where,  for  perhaps  the  first  time  in 
the  history  of  computing,  scientific  research  played  a  central  role  in  driving  the  cre- 
ation of  hardware  and  software  technology  rather  than  being  only  at  the  receiving 
end.  Perhaps  more  than  anything  else,  however,  the  NSF  national  centers  became 
a  focal  point  for  illustrating  the  vital  importance  of  the  basic  research  enterprise 
to  the  overall  well-being  of  the  United  States. 

The  NSF  supercomputing  centers  program  in  general,  and  the  Pittsburgh 
Supercomputing  Center  in  particular,  have  played  a  prominent  role  in  my  own  re- 
search, and  I  wish  to  share  how  a  partnership  between  the  PSC  and  the  NSF  STC 
I  direct  is  poised  to  help  re-define  numerical  weather  prediction.  The  goal  of  my 
STC,  which  like  the  supercomputer  centers  is  subject  to  a  sunset  law,  is  to  develop 
and  operationally  demonstrate  techniques  for  the  practical  numerical  prediction  of 
individual  spring  and  wintertime  storms  up  to  6  hours  in  advance.  While  other 
groups  around  the  world  are  working  on  various  components  of  this  important  prob- 
lem, CAPS  is  prototyping  the  entire  prediction  process  in  collaboration  with  the  Na- 
tional Weather  Service  and  other  groups.  This  prototyping  involves  data  gathering, 
numerical  forecasting,  and  forecast  product  development,  dissemination,  and  evalua- 
tion. The  entire  effort,  which  started  in  1989,  focuses  on  large  pieces  of  software, 
high-speed  communications  networks,  and  powerful  computers  to  forecast  the 
weather. 

To  understand  the  value  of  CAPS'  interaction  with  PSC,  it  should  be  recognized 
that  operational  numerical  weather  prediction  is  now  conducted  almost  exclusively 
at  the  national  or  multi-national  level,  wherein  a  centralized  facility  collects  and 
processes  all  relevant  observational  data,  runs  a  suite  of  numerical  models  ranging 
in  scale  from  regional  to  global,  and  generates  and  disseminates  forecasts  and  relat- 
ed products,  often  targeted  to  specific  needs  such  as  aviation.  None  of  the  models 
is  currently  able  to  represent  explicitly  individual  thunderstorms  or  storm  com- 
plexes. The  desire  to  predict  such  events  with  a  lead  time  of  a  few  hours  is  obvious, 
as  thunderstorms  and  their  related  weather  are  principally  responsible  for  flash 
floods,  damaging  surface  winds,  hail,  tornadoes,  and  low-level  turbulence  that  poses 
a  threat  to  aircraft.  Indeed,  the  economic  impact  of  improved  predictions  of  such 
events  is  staggering.  Commercial  transportation  companies,  power  and  communica- 
tion utilities,  agricultural  interests,  public  recreation  facilities,  construction  compa- 
nies, and  a  host  of  other  private  and  government  entities  lose  millions  of  dollars 
each  year  as  a  result  of  unanticipated  weather  events  that  occur  on  scales  smaller 

than  can  be  predicted  using  current  technology.  In  the  case  of  the  Oklahoma  City 
bombing,  recovery  efforts  could  have  been  aided  considerably  by  accurate  short-term 

(6-hour)  numerical  forecasts  of  precipitation,  winds,  and  lightning. 
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The  ability  of  CAPS  to  accomplish  its  mission  depends  critically  upon  the  effective 
use  of  high-performance  computing  and  communications  systems.  Because  thunder- 
storms, for  example,  have  relatively  short  lifetimes  (a  few  hours)  compared  to  larg- 
er-scale weather  systems  (a  few  days),  their  associated  numerical  forecasts  must  be 
generated  and  disseminated  to  the  public  very  quickly — 50  to  100  times  faster  than 
the  weather  evolves — to  be  of  practical  value.  This  poses  an  extreme  computational 
challenge  that  is  uniquely  suited  for  exploration  at  the  NSF  national  centers. 

As  elements  of  the  CAPS  program  matured,  a  series  of  yearly,  realtime  oper- 
ational evaluations  of  its  prediction  system  was  begun  in  1993  over  the  Southern 
Great  Plains.  Only  through  such  tests,  which  provide  first-hand  exposure  to  the  rig- 
ors and  time  constraints  of  an  operational  environment,  can  one  truly  identify  the 
strengths  and  weaknesses  of  the  prediction  system.  Furthermore,  such  tests  are 
vital  for  determining  the  most  effective  use  of  available  computers — an  important 
logistical  and  economic  consideration  for  the  National  Weather  Service.  Options 
range  from  conducting  the  predictions  at  a  single  centraUzed  facility  to  using  a 
number  of  smaller  distributed  facilities  around  the  country.  It  is  a  goal  of  our 
project  with  PSC  to  provide  information  to  effectuate  this  decision,  and  it  is  our  be- 
lief that  this  would  not  have  been  possible  apart  from  the  NSF  supercomputing  cen- 
ters. 

The  specific  operational  tests  being  conducted  by  CAPS  at  the  PSC  involve  dedi- 
cated use  of  their  Cray  T3D  (the  entire  machine,  or  512  processors)  and  partial  dedi- 
cated use  of  their  Cray  C90  for  4  to  6  hours  each  day  during  the  operational  test 
period,  which  runs  from  April  15  to  June  15  each  year.  The  numerical  forecast 
model  is  run  using  available  realtime  observations,  including  those  from  the  new 
national  network  of  Doppler  radars,  and  6-hour  forecasts  are  generated  and  exam- 
ined by  students,  scientists  and  forecasters  both  in  Norman,  OMahoma  and  around 
the  world,  in  the  latter  case  via  images  placed  automatically  on  our  World  Wide 
Web  page.  To  facilitate  this  complex  effort  that  involves  approximately  50  scientists 
and  students  at  CAPS,  PSC  dedicates  two  technical  support  personnel  to  our  project 
for  the  duration  of  the  tests,  and  assigns  several  others  as  consultants  to  deal  with 
a  variety  of  logistical  issues.  By  working  closely  with  PSC  in  this  manner  and  being 
allowed  to  use  dedicated  computer  resources,  CAPS  is  able  to  prototype  a  variety 
of  computational  strategies  for  possible  implementation  by  the  National  Weather 
Service  early  in  the  next  century. 

The  results  of  the  1995  spring  operational  test  forecasts  were  so  encouraging  that 
a  major  commercial  U.S.  airline  is  in  the  final  stages  of  negotiating  a  3-year,  $1  mil- 
lion research  and  development  contract  with  CAPS  to  continue  evolving  the  small- 
scale  numerical  prediction  technology  and  to  evaluate  its  viability  in  daily  airline 
operations.  This  private  sector  involvement,  which  brings  additional  support  to  two 
federally-funded  centers  (PSC  and  CAPS),  would  not  have  been  possible  without  the 
dedicated  supercomputer  time  and  extensive  scientific  collaboration  offered  by  the 
PSC.  Indeed,  the  CAPS-PSC  collaboration,  which  began  in  1993,  provides  a  glimpse 
at  the  type  of  partnerships  and  strategic  alliances  NSF  proposes  to  create  through 
the  re-competition.  Further,  it  aptly  illustrates  the  private  sector's  interest  in  and 
willingness  to  fund  university-based  research  and  development  efforts. 

By  every  reasonable  measure,  the  NSF  supercomputing  centers  are  a  resounding 
success  and  have  played  a  significant  role  in  moving  the  United  States  forward  as 
a  strong  competitor  on  the  global  scale.  It  is  not  inappropriate,  however,  after  a  dec- 
ade or  so,  to  revisit  the  intent  and  structure  of  this  type  of  program.  Indeed,  this 
philosophy  of  re-examination  underpins  many  of  the  NSF's  successful  center-ori- 
ented programs,  and  should  be  continued. 

The  proposed  restructuring  is  taking  place  in  an  environment  significantly  dif- 
ferent than  existed  in  1985,  when  the  original  5  supercomputing  centers  were  initi- 
ated. The  scientific  community  is  far  more  computer  literate,  and  scientists  and  en- 
gineers are  crossing  disciplines,  working  on  linked  problems,  and  tackling  more 
challenging  projects  than  ever  before.  Most  importantly,  however,  is  the  prolifera- 
tion of  powerful,  inexpensive  mid-range  computers  which  has  placed  significant  com- 
puting power  in  scientist's  laboratories  and  on  their  desktops.  In  this  regard,  the 
original  center  concept  is  losing  its  effectiveness — quite  appropriately  so,  I  might 
add,  and  those  projects  now  capable  of  being  performed  uniquely  at  the  national 
NSF  centers  are  the  type  that  require  specialized  or  dedicated  use  of  the  most  pow- 
erful resources  available  and  close  technical  and  scientific  support  for  extended  peri- 
ods of  time. 

In  this  regard,  the  concept  for  the  center  re-competition  is,  in  my  view,  a  good 
one.  Specifically,  I  believe  a  need  will  continue  to  exist  for  a  few  high-capacity 
supercomputing  centers  that  provide  the  most  powerful  machines  and  the  most  ad- 
vanced technology  commercially  available.  Given  the  distributed  nature  of  both 
science  and  the  national  information  infrastructure,  it  is  appropriate  that  these  few 
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major  centers  be  augmented  by  a  number  of  strategic  partners  or  alliances,  each  of 
which  has  a  specific  application  area  that  both  provides  special  facilities  and  has 
needs  that  require  significant  ties  to  a  major  NSF  supercomputing  center.  The  con- 
cept is  somewhat  analogous  to  a  central  library  and  its  branches;  each  has  its  dis- 
tinct advantages  and  purpose,  but  neither  can  perform  the  entire  mission  on  its 
own. 

Care  must  be  taken  in  evolving  this  infrastructure,  however,  so  as  to  avoid  two 
potential  problems:  a  reduction  in  overall  resources  available  to  the  national  com- 
munity, and  elimination  of  diversity  among  the  centers.  With  regard  to  the  former, 
I  believe  that  if  the  current  4  centers  are  consolidated  into  2  or  3,  they  might  well 
lose  their  effectiveness  and  become  mass  production  facilities  ("mega  centers")  of  the 
type  that  could  easily  be  provided  by  the  private  sector.  This  is  not  desirable.  In 
my  view,  the  success  of  the  new  partnership  concept  rests  upon  maintaining  a  prop- 
er balance  between  the  size  of  each  major  center  (at  a  university)  and  the  total 
number  of  such  centers.  Consequently,  any  reduction  in  overall  funding  for  the  cen- 
ters program  would  be  a  mistake  and  would  undoubtedly  slow  the  pace  of  our  sci- 
entific research  as  well  as  diminish  our  ability  to  move  science  and  technology  into 
arenas  that  directly  benefit  society. 

With  regard  to  the  second  point  above,  it  is  important  to  recognize  and  preserve 
the  diversity  that  now  exists  among  the  supercomputing  centers.  The  new  centers 
should,  like  the  old,  be  provided  sufficient  latitude  to  establish  their  own  identities, 
developing  specific  areas  of  interest  and  expertise.  Each  center  cannot  and  should 
not  be  expected  to  be  all  things  to  all  people.  Indeed,  a  distinct  and  significant  ad- 
vantage of  CAPS'  interaction  with  PSC  is  the  personal  and  extensive  attention  pro- 
vided to  CAPS'  scientists  as  well  as  PSCs  specific  interests  in  the  technology  re- 
quired for  distributed  operational  weather  prediction. 

The  re-competition  process  itself  seems  sound,  and  I  support  the  plan  for  an  ini- 
tial 5-year  award  followed  by  a  5-year  option.  I  have  found  this  timescale  to  be  quite 
appropriate  for  the  STC  program.  I  also  have  great  respect  for  sunset  provisions  and 
believe  they  should  be  applied  to  the  new  centers. 

Mr.  SCHIFF.  Dr.  Gale,  please. 

STATEMENT  OF  DR.  DOUGLAS  S.  GALE,  ASSISTANT  VICE 
PRESIDENT  FOR  INFORMATION  SYSTEMS  AND  SERVICES, 
THE  GEORGE  WASHINGTON  UNIVERSITY 

Dr.  Gale.  Thank  you,  Mr.  Chairman,  and  Members  of  the  Com- 
mittee, for  the  opportunity  to  testify: 

I  would  like  to  note  for  the  record  that,  contrary  to  stereotypes, 
the  academic  panel  generated  the  least  amount  of  paper  of  any  of 
the  three  panels  today. 

My  name  is  Douglas  Gale.  I  am  Assistant  Vice  President  for  In- 
formation Systems  and  Service,  and  Professor  of  Physics  at  The 
George  Washington  University. 

Prior  to  this  position,  I  was  a  Professor  of  Physics  and  Computer 
Science  at  the  University  of  Nebraska-Lincoln.  I  was  also  the  Di- 
rector of  Computing  and  served  as  one  of  the  founders  of  the  NSF 
Net,  the  precursor  of  today's  Internet,  and  served  as  President  of 
MidNet,  a  regional  network  on  the  NSF  Net,  before  its  spinoff  as 
a  successful  for-profit  corporation  in  1994. 

My  comments  today  will  focus  on  two  areas: 

The  accomplishments  of  the  National  Science  Foundation  High 
Performance  Computing  and  Communications  Initiatives,  and  an 
assessment  of  the  recompetition  process. 

Over  the  past  decade  the  high-performance  computing  and  com- 
munications activities  of  the  National  Science  Foundation  have 
been  remarkably  successful. 

They  provide  a  textbook  example  of  how  federal  funds  can  be 
used  to  prime  the  technology  pump  and  leverage  modest  govern- 
ment investments. 
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In  the  early  and  mid  1980s,  computer  communications  tech- 
nology was  primitive  and  it  was  used  primarily  by  a  small  group 
of  technies. 

In  December  of  1995,  the  cover  of  Newsweek  proclaimed  1995  as 
"The  Year  of  The  Internet".  The  growth  of  the  InterNet  is  largely 
a  result  of  the  NSF  High  Performance  Computing  and  Communica- 
tions Initiatives. 

While  it  is  difficult  to  measure  the  economic  impact  of  the  NSF 
High  Performance  Computing  and  Communications  Initiatives, 
there  are  many  anecdotal  examples. 

The  United  States  now  has  a  multimillion  dollar  router  industry 
that  dominates  the  world  market.  In  1986,  that  industry  was  al- 
most  

Mr.  SCHIFF.  I  am  sorry.  Dr.  Gale?  What  industry?  I  did  not  hear 
the  word. 

Dr.  Gale.  The  "router."  It  is  a  technical  box  that  is  used  in  com- 
puter communications  and  is  now  a  million  dollar  industry,  and  we 
dominate  that  world  market.  In  1986  it  did  not  even  exist. 

The  word  is  still  not  found  in  the  dictionary  in  this  context. 

The  NSF  initiatives  provided  many  of  the  first  customers  for  that 
fledgling  industry  and  helped  set  the  technical  parameters. 

The  cost  to  the  taxpayers  for  these  initiatives  was  modest.  My 
experience  in  building  campus  and  regional  networks  indicates  that 
for  every  dollar  spent  by  the  Federal  Government,  $10  was  spent 
by  education  and  their  corporate  supporters. 

The  spinoff  commercial  utilization  of  this  technology  is  privately 
funded. 

I  would  like  to  provide  the  Committee  with  an  example  of  how 
the  NSF  Initiatives  have  helped  contribute  practical  benefits  to  the 
American  people. 

The  National  Crash  Analysis  Center  at  The  George  Washington 
University's  Virginia  campus  develops  computer  models  of  real  cars 
and  trucks  and  then  crashes  them  in  a  computer  to  determine 
what  would  happen  to  computer  models  of  test  dummies. 

The  accuracy  of  the  process  is  verified  by  comparing  the  simula- 
tion with  real  crash  results.  These  simulations  help  in  the  design 
of  safer  vehicles  and  roadside  objects.  The  Center  is  being  sup- 
ported and  widely  used  by  the  automotive  industry. 

As  computing  and  communications  capabilities  grow,  we  are 
moving  from  a  computer  model  of  the  test  dummy  to  a  far  more 
accurate  model  of  the  human  body  based  upon  extensive  data  as- 
sembled by  the  National  Institutes  of  Health.  This  will  allow  sim- 
ulation not  just  of  the  impact  forces  on  the  surface  of  the  test 
dummy,  but  the  simulation  of  the  effects  of  a  crash  on  human 
bones  and  internal  organs. 

You  cannot  do  these  kinds  of  simulations  on  your  PC.  They  re- 
quire the  kinds  of  high-performance  computing  hardware  and  soft- 
ware and  communications  that  has  been  developed  at  the 
Supercomputing  Centers.  They  require  scientists  and  engineers 
trained  in  the  use  of  high-  performance  computing  and  communica- 
tion, the  kind  of  training  provided  by  the  NSF  Initiatives. 

The  second  focus  of  my  comments  is  an  assessment  of  the 
recompetition  process.  In  my  experience,  the  NSF  peer  review  proc- 
ess is  both  fair  and  insightful. 
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In  1985,  I  submitted  an  unsolicited  proposal  to  the  NSF  to  de- 
velop and  deploy  a  regional  computer  network  to  provide  access  to 
the  NSF's  Supercomputing  Centers.  At  that  time  there  were  no 
such  regional  networks,  and  I  was  a  relatively  unknown  researcher 
and  administrator  from  an  institution  located  in  the  rural  Midwest. 

After  a  peer  review  process,  I  received  funding.  As  I  become  more 
involved  with  that  process  and  have  participated  in  peer  reviews 
myself,  I  become  even  more  impressed  with  the  ability  of  the  proc- 
ess to  distribute  resources  to  those  projects  that  offer  the  greatest 
potential. 

I  am  currently  working  with  the  Southeastern  University's  Re- 
search Association  to  develop  a  response  to  the  Partnerships  for 
Advanced  Computational  Infrastructure  solicitation. 

We  have  found  the  playing  field  to  be  very  level.  The  regional 
question  and  answer  sessions  that  followed  the  original  solicitation 
have  been  thorough  and  fair.  The  use  of  the  Worldwide  Web  post- 
ing information  about  the  solicitation  is  particularly  innovative  and 
should  be  commended. 

While  there  are  concerns  within  the  user  community  about  pos- 
sible changes  in  existing  funding  for  individual  projects  and  the 
amount  of  funding  available,  there  is  in  general  widespread  enthu- 
siasm for  the  opportunity  to  explore  new  ideas,  collaborations,  smd 
partnerships. 

Within  the  Southeastern  University  Research  Association  there 
have  been  numerous  meetings  and  workshops  in  which  we  have  re- 
evaluated what  we  do,  how  we  do  it,  with  whom  we  do  it,  and  most 
importantly  why  we  do  it. 

Those  discussions  have  resulted  in  new  and  innovative  ap- 
proaches not  only  to  scientific  research  itself,  but  also  of  how  that 
research  can  be  used  to  benefit  the  region  and  the  Nation. 

In  short,  we  have  already  seen  benefits  from  the  solicitation. 

In  closing  let  me  restate  my  two  main  points: 

Over  the  past  decade  the  high-performance  computing  and  com- 
munications activities  of  the  National  Science  Foundation  have 
been  remarkably  successful.  The  user  community  views  the  NSF 
peer  review  process  as  both  fair  and  insightful,  and  looks  forward 
to  the  opportunities  provided  by  the  new  solicitation. 

Again  I  want  to  thank  you  for  the  opportunity  to  testify,  and 
would  be  happy  to  respond  to  any  questions. 

[The  prepared  statement  of  Dr.  Gale  follows:] 

Testimony  of  Dr.  Douglas  S.  Gale 

AssT.  Vice  President  Information  Systems  and  Services 

The  George  Washington  University 

before  the 

House  Subcommittee  on  Basic  Research  of  the  Committee  on  Science 

March  19,  1996 

Thank  you  Mr.  Chairman  and  members  of  the  committee  for  the  opportunity  to 
present  testimony.  My  name  is  Douglas  Gale.  I  am  Assistant  Vice  President  for  In- 
formation Systems  and  Services  at  The  George  Washington  University.  Prior  to  this 
position,  I  was  a  faculty  member  in  the  Department  of  Physics  and  the  Department 
of  Computer  Science  as  well  as  the  Director  of  Computing  at  the  University  of  Ne- 
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braska-Lincoln.  I  was  also  one  of  the  founders  of  the  NSFNET,  the  precursor  to  to- 
day's Internet,  and  served  as  the  President  of  MIDnet,  a  regional  network  in  the 
NSFNET,  before  it  spun  off  as  a  successful  for-profit  corporation  in  1994. 

My  comments  today  will  focus  on  two  areas:  the  accomplishments  of  the  National 
Science  Foundation  (NSF)  high  performance  computing  and  communications  initia- 
tives and  an  assessment  of  the  re-competition  process. 

Over  the  past  decade,  the  high  performance  computing  and  communications  ac- 
tivities of  the  NSF  have  been  remarkably  successful.  They  provide  a  textbook  exam- 
ple of  how  federal  funds  can  be  used  to  prime  the  technology  pump  and  leverage 
modest  government  investments. 

In  the  early  and  mid  80's  computer  communications  technology  was  primitive  and 
used  by  a  relatively  small  number  of  "techies."  By  December  of  1995,  the  cover  of 
Newsweek  proclaimed  1995  as  "The  Year  of  the  Internet."  The  growth  of  the 
Internet  is  largely  the  result  of  NSF  high  performance  computing  and  communica- 
tions initiatives.  For  example,  the  original  development  of  the  software  that  is  used 
to  access  the  popular  "World  Wide  Web"  was  developed  at  one  of  the  NSF 
Supercomputer  Centers.  That  effort  has  spawned  a  new  industry  that  now  employs 
thousands. 

While  it  is  difficult  to  measure  the  economic  impact  of  the  NSF  high  performance 
computing  and  communications  activities,  there  are  many  anecdotal  examples.  The 
United  States  now  has  a  multi-million  dollar  "router"  industry  that  dominates  the 
world  market.  In  1986  that  industry  was  almost  non-existent.  The  NSF  initiatives 
provided  many  of  the  first  customers  for  that  fledgling  industry  and  helped  to  set 
the  technical  parameters. 

The  cost  to  the  taxpayers  for  these  initiatives  was  modest.  My  experience  in  build- 
ing campus  and  regional  networks  indicates  that  for  every  dollar  spent  by  the  fed- 
eral government,  ten  dollars  was  spent  by  education  and  their  corporate  supporters. 
The  spin-off  commercial  utilization  of  this  technology  is  privately  funded. 

I  would  like  to  provide  the  Committee  with  an  example  of  how  the  NSF  initiatives 
have  helped  contribute  practical  benefits  to  the  American  people.  The  National 
Crash  Analysis  Center  at  The  George  Washington  University's  Virginia  Campus  de- 
velops computer  models  of  real  cars  and  trucks  and  then  "crashes '  them  in  a  com- 
puter to  determine  what  would  happen  to  computer  models  of  test  dummies.  The 
accuracy  of  the  process  is  verified  by  comparing  the  simulation  with  real  crash  re- 
sults. These  simulations  help  in  the  design  of  safer  vehicles  and  road  side  objects. 
The  Center  is  currently  funded  by  the  Federal  Highway  Administration  (FHWA) 
and  the  National  Highway  Traffic  Safety  Administration  (NHTSA)  as  well  as  being 
supported  and  widely  used  by  the  automotive  industry. 

As  computing  and  communication  capability  grows,  we  are  moving  from  a  com- 
puter model  of  a  test  dummy  to  a  far  more  accurate  model  of  the  human  body  based 
upon  extensive  data  assembled  by  the  National  Institutes  of  Health.  This  will  allow 
simulation  not  just  of  the  impact  forces  on  the  surfaces  of  a  test  dummy,  but  simula- 
tion of  the  effect  of  a  crash  on  internal  human  bones  and  organs. 

You  can't  do  these  kinds  of  simulations  on  your  PC.  They  require  the  kinds  of 
high  performance  computing  hardware  and  software  and  communications  that  has 
been  developed  at  the  Supercomputer  Centers.  They  require  scientists  and  engi- 
neers trained  in  the  use  of  high  performance  computing  and  communications — the 
kind  of  training  provided  by  the  NSF  initiative. 

The  second  focus  of  my  comments  is  an  assessment  of  the  re-competition  process. 

In  my  experience,  the  NSF  peer  review  process  is  both  fair  and  insightful.  In  1985 
I  submitted  an  unsolicited  proposal  to  the  NSF  to  develop  and  deploy  a  regional 
computer  network  to  provide  access  to  the  NSF  Supercomputer  Centers.  At  chat 
time  there  were  no  such  regional  networks  and  I  was  a  relatively  unknown  re- 
searcher and  administrator  from  an  institution  located  in  the  rural  midwest.  After 
a  peer  review  process,  I  received  funding.  As  I  have  become  more  involved  with  the 
process  and  have  participated  in  peer  reviews  myself,  I  have  become  even  more  im- 
pressed with  the  ability  of  the  process  to  distribute  resources  to  those  projects  that 
offer  the  greatest  potential. 

I  am  currently  working  with  the  Southeastern  Universities  Research  Association 
(SURA)  to  develop  a  response  to  the  Partnerships  for  Advanced  Computational  In- 
frastructure Solicitation.  We  have  found  the  playing  field  to  be  extremely  level.  The 
regional  "question  and  answer"  sessions  that  followed  the  original  solicitation  have 
been  thorough  and  fair.  The  use  of  the  World  Wide  Web  to  post  information  about 
the  solicitation  is  particularly  innovative  and  should  be  commended. 

While  there  are  concerns  within  the  user  community  about  possible  changes  in 
existing  funding  for  individual  projects  and  the  amount  of  funding  available,  there 
is  widespread  enthusiasm  for  the  opportunity  to  explore  new  ideas,  collaborations, 
and  partnerships.  Within  SURA,  there  have  been  numerous  meetings  and  work- 
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shops  in  which  we  have  re-evaluated  what  we  do,  how  we  do  it,  with  whom  we  do 
it,  and  most  importantly,  why  we  do  it.  These  discussions  have  resulted  in  new  and 
innovative  approaches  not  only  to  scientific  research  itself,  but  also  how  that  re- 
search can  be  used  to  benefit  the  region  and  nation. 

In  closing,  let  me  restate  my  two  main  points.  Over  the  past  decade,  the  high  per- 
formance computing  and  communications  activities  of  the  NSF  have  been  remark- 
ably successful.  The  user  community  views  the  NSF  peer  review  process  as  both  fair 
and  insightful  and  looks  forward  to  the  opportunities  provided  by  the  new  solicita- 
tion. 

Again,  I  want  to  thank  you  for  the  opportunity  to  testify  and  will  be  happy  to 
respond  to  any  questions. 

Mr.  SCHIFF.  Thank  you,  Dr.  Gale. 

I  have  one  question.  I  beheve  you  have  all  touched  upon  it,  but 
I  just  want  to  make  sure  it  is  clear.  That  is,  you  know  the  current 
plan  to  revamp  how  we  are  handling  supercomputing  through  the 
NSF  involves  a  number  of  factors,  including  the  possible 
downsizing  of  four  centers  to  two  centers. 

Now  I  add  that,  if  that  happens,  it  happens  with  other  changes 
in  the  system  that  have  been  discussed. 

I  just  want  to  ask  again.  On  balance,  if  the  revamping — well, 
maybe  I  should  just  leave  it  there — on  balance,  is  the  revamping 
as  proposed,  are  you  favorable  towards  it,  or  unfavorable  towards 
it? 

And  again  I  know  I  may  be  asking  you  to  repeat,  but  I  would 
like  a  precise  answer  to  that  question. 

Dr.  Vernon,  may  I  call  upon  you  first,  please? 

Dr.  Vernon.  Well,  I  think  I  was  pretty  clear,  but  I  will  restate 
it. 

Mr.  SCHIFF.  I  think  you  were,  too. 

Dr.  Vernon.  I  am  in  favor  of  the  proposed  restructuring. 

Mr.  SCHIFF.  Dr.  Droegemeier? 

Dr.  Droegemeier.  Yes,  I  am  also  in  favor.  I  would  just  add  a 
cautionary  note  that  I  think  that  you  run  into  some  potential  dif- 
ficulty with  bringing  the  number  of  centers  down  say  even  to  two 
I  think  would  be  right  on  the  edge  of  concern.  For  example,  if  one 
failed  or  something  like  that,  but  I  have  a  grave  concern  that  if  you 
do  consolidate  and  create  two  much  larger  centers,  I  think  their 
ability  to  deal  with  groups  individually  becomes  diminished. 

That  is  what  I  would  worry  about.  I  kind  of  liken  that  to  doing 
banking  at  a  small  bank  versus  an  international  conglomerate  or 
something  like  that.  You  get  more  personalized  attention  and  it  can 
be  much  more  effective  if  the  centers  are  not  incredibly  large. 

Mr.  SCHIFF.  Dr.  Gale? 

Dr.  Gale.  On  balance,  the  Southeastern  University  Research  As- 
sociation supports  the  NSF  Initiative.  We  feel  that,  while  there 
may  be  some  consolidation  of  the  leading  edge  centers,  the  solicita- 
tion is  worded  in  such  a  way  that  there  will  actually  be  more  part- 
nerships and  more  distribution  of  resources  than  we  currently  see. 

I  think  this  is  being  borne  out  by  the  experience  of  those  of  us 
that  are  putting  together  proposals.  We  are  already  creating  new 
alliances,  new  partnerships  that  did  not  exist  before. 

Mr.  SCHIFF.  Thank  you. 

That  is  all  the  questions  I  have. 

Mr.  Doyle? 
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Mr.  Doyle.  Well  it  is  just  you  and  me,  Mr.  Chairman.  I  have 
just — just  to  sum  up,  Dr.  Droegemeier,  what  led  you  to  go  to  the 
Pittsburgh  Center  as  opposed  to  the  other  three? 

Dr.  Droegemeier.  Could  I  ask  a  question  on  top  of  that?  The 
question  may  be,  that  would  proceed  that,  is  why  are  we  not  com- 
peting at  the  National  Center  for  Atmospheric  Research,  which  has 
been  around  for  a  much  longer  time,  which  we  were  initially  but 
as  a  testimony  I  think  to  the  success  of  the  NSF  Centers,  when 
they  began  in  1985  they  started  with  a  clean  slate,  no  historical 
baggage  or  anj^thing  like  that,  and  they  charged  ahead  and  they 
blew  right  past  NCAR,  for  example,  not  because  NCAR  was  inept 
but  just  because  they  had  had  a  history  and  were  doing  things  a 
certain  way,  and  it  took  them  a  long  time  to  catch  up. 

We  had  computed  at  NCSA  for  a  few  years,  and  then  the  Alloca- 
tions Review  Board  actually  moved  us  to  Pittsburgh  because  what 
we  were  doing  was  a  little  bit  more  appropriate  to  Pittsburgh. 

We  actually  affected — and  I  got  my  degree  at  the  University  of 
Illinois,  so  Larry  might  be  sitting  back  there  wondering  why  I  am 
not  computing  there 

[Laughter.] 

Dr.  Droegemeier.  (continuing)  — but  what  we  had,  and  we  shift- 
ed to  Pittsburgh,  and  pretty  much  when  you  start  one  place  you 
continue,  but  we  have  had  every  good  success  with  Pittsburgh,  I 
think  again  because  of  their  size.  They  have  worked  closely  with 
us  as  scientific  collaborators  in  every  true  sense  of  the  phrase  to 
where  when  we  do  our  prediction  runs  and  things  like  that  we  ac- 
tually exchange  people. 

We  talk  with  these  people  daily. 

We  are  sort  of  prototyping  an  actual  operational  forecast  environ- 
ment. I  think  the  size  of  the  center  and  the  interest  of  their  sci- 
entists in  the  kind  of  distributed  computing  that  we  are  doing,  the 
parallel  computing,  and  especially  the  real-time  aspect  of  this  thing 
which,  like  I  say,  if  you  can't  do  it  far  ahead  of  the  weather  you 
might  as  well  not  do  it  at  all,  has  been  a  very  specific  interest  of 
the  Pittsburgh  Center.  So  it  has  been  a  nice  marriage  in  that 
sense. 

Mr.  Doyle.  Very  good.  I  guess  I  would  just  echo  your  concerns 
that  all  of  us  here  want  to  see  what  has  been  a  very  successful  pro- 
gram continue  and  want  to  support  the  best  way  to  make  sure  that 
that  does  continue. 

I  guess  when  I  looked  at  your  testimony  and  Dr.  Vernon's  that, 
while  there  is  a  concern  that  we  may  not,  due  to  budgetary  consid- 
erations, be  able  to  continue  to  support  many  centers,  that  there 
also  is  a  concern  that  if  we  reduce  the  leading  edge  centers  to  one 
or  two  that  that  also  brings  a  set  of  problems  that  we  need  to  be 
cognizant  of. 

So  I  thank  you  all  for  your  testimony. 

Thank  you,  Mr.  Chairman. 

Mr.  SCHIFF.  Well,  I  want  to  say  that,  as  evidenced  here,  last  was 
definitely  not  least.  We  appreciate  your  patience.  Someone  has  to 
be  a  first  panel,  and  someone  has  to  be  a  third  panel. 

We  appreciate  your  being  here  this  long.  We  appreciate  your  con- 
tribution. And  with  that,  the  Subcommittee  is  in  recess. 
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[Whereupon,  at  4:28  p.m.,  March  19,  1996,  the  hearing  was  ad- 
journed.] 

[The  following  material  was  received  for  the  record:] 

APPENDIX 

Honorable  Steve  Schiff 
Hearing — Partnership  for  Advanced  Computational  Infrastructure  Program 

March  19,  1996 

Today,  the  Subcommittee  will  exercise  its  oversight  responsibilities  for  the  Na- 
tional Science  Foundation  by  receiving  testimony  on  the  solicitation  of  the  Partner- 
ship for  Advanced  Computational  Infrastructure  Program. 

As  this  Congress  continues  to  find  new  ways  to  balance  the  budget  and  reduce 
the  size  of  government,  agencies  are  looking  to  maximize  the  value  of  each  taxpayer 
dollar.  Of  the  government  agencies,  the  National  Science  Foundation  is  one  of  the 
best  federal  agencies  at  running  a  lean  and  efficient  organization.  I  extend  my  com- 
pliments to  Dr.  Lane  and  all  his  employees. 

This  Congress  provided  NSF  with  an  increase  in  FY  96  funding  over  FY  95.  But, 
hard  choices  must  still  be  made  on  funding  priorities  and  the  direction  programs 
should  be  taking  as  we  head  into  the  21st  Century.  One  of  those  programs  under- 
going review  is  NSF's  Supercomputer  Center  program.  NSF  is  making  tough  deci- 
sions on  the  direction  of  the  supercomputer  centers,  their  funding,  and  what  role 
they  should  play  in  research  applications. 

The  Partnership  for  Advanced  Computational  Infrastructure  Program  builds  on 
and  replaces  the  current  NSF  Supercomputer  Centers  Program  established  in  1985. 
The  new  program  will  focus  on  taking  advantage  of  newly  emerging  opportunities 
in  high  performance  computing  and  communications. 

The  Subcommittee  is  interested  in  assessing  the  major  accomplishments  of  the 
NSF  Supercomputing  Centers  Program  over  the  past  ten  years  as  well  as  the  pro- 
posed restructuring  of  the  program.  Additionally,  the  Subcommittee  desires  to  re- 
ceive any  comments  or  specific  suggestions  from  witnesses  on  the  recompetition 
process  which  NSF  has  developed  for  the  program. 

To  outline  a  brief  history  of  the  supercomputer  program,  the  NSF  Supercomputing 
Centers  program  was  established  in  1984  following  strong  expressions  of  the  need 
for  such  computing  resources  for  the  academic  research  community.  During  a  review 
in  1990,  there  was  a  distinct  effort  to  expand  outreach  services,  with  initial  efforts 
intended  to  forge  closer  ties  to  industries  that  could  profit  from  exposure  to  high 
performance  computing,  and  to  include  the  community  at  large. 

The  Task  Force  report  we  will  receive  testimony  on  today  is  the  latest  of  many 
studies  of  the  Centers.  The  Director  of  the  National  Science  Foundation  established 
the  "Task  Force  on  the  Future  of  the  NSF  Supercomputer  Centers  Program"  in  De- 
cember 1994.  The  Task  Force  was  asked  ".  .  .  to  analyze  various  alternatives  for 
the  continuation,  restructuring,  or  phase-out  of  NSF's  current  Supercomputer  Cen- 
ter program,  or  the  development  of  similar  future  program(s),  and  to  make  rec- 
ommendations among  the  alternatives." 

In  January,  the  National  Science  Board  adopted  the  recommendation  for  a  new 
competition  of  the  Centers  and  NSF  announced  the  Partnership  for  Advanced  Com- 
putational Infrastructure.  Preliminary  solicitations  are  due  April  15,  1996.  The  final 
awards  will  be  announced  in  the  fall  of  1996. 

To  learn  more  about  the  Partnership  program,  it  is  a  pleasure  to  welcome  Dr. 
Paul  Young,  the  Assistant  Director  for  Computer  and  Information  Science  and  Engi- 
neering at  the  National  Science  Foundation.  We  also  welcome,  Dr.  Ed  Hayes,  Chair- 
man of  the  Task  Force  on  the  Future  of  the  NSF  Supercomputer  Centers  Program, 
who  recommended  the  National  Science  Board  have  a  recompetition  of  the  existing 
supercomputer  centers. 

On  our  second  panel,  we  will  welcome  the  directors  of  the  4  Centers:  Dr.  Malvin 
Kalos,  of  the  Cornell  Theory  Center:  Dr.  Larry  Smarr,  of  the  National  Center  for 
Supercomputing  Applications;  Dr.  Douglas  Pewitt,  of  the  San  Diego  Supercomputer 
Center;  and  from  the  Pittsburgh  Supercomputer  Center,  we  have  Dr.  Michael  Le- 
vine  and  Dr.  Ralph  Roskies. 

And  on  our  third  panel,  we  will  hear  from  supercomputer  researchers,  Dr.  Mary 
Vernon  and  Dr.  Kelvin  Droegemeir.  We  will  hear  also  from  Dr.  Douglas  Gale  who 


205 

will  present  the  views  of  organizations  which  want  to  be  involved  in  the  new 
supercomputer  program  at  NSF. 


OPENING  STATEMENT 

HEARING  ON 

PARTNERSHIP  FOR  ADVANCED  COMPUTATIONAL  INFRASTRUCTURE 

PROGRAM 

BY 

THE  HONORABLE  BUD  CRAMER  (D-AL) 

RANKING  DEMOCRATIC  MEMBER 
SUBCOMMITTEE  ON  BASIC  RESEARCH 

March  19,  1996 

I  am  pleased  to  join  the  Chairman  in  welcoming  our  distinguished  witnesses  this 
afternoon,  and  I  congratulate  him  on  calling  a  hearing  on  this  important  subject — 
the  future  of  the  National  Science  Foundation's  Supercomputer  Centers. 

Today  we  are  reviewing  an  NSF  program  which  this  Committee  has  supported 
and  encouraged  since  its  inception  over  10  years  ago.  Hearings  in  the  early  1980s 
highlighted  concerns  from  the  academic  research  community  that  researchers  had 
very  limited  access  to  advanced  scientific  computers.  This  was  of  concern  because 
high  performance  computers  were  beginning  to  show  enormous  promise  as  tools  for 
attacking  previously  intractable  problems.  They  had  opened  entirely  new  areas  of 
scientific  inquiry. 

At  the  outset,  the  Supercomputer  Centers  program  simply  provided  computer 
time  to  academic  researchers  at  a  few  existing  supercomputer  sites.  The  program 
was  subsequently  enlarged  in  scope  to  accelerate  the  development  and  use  of  new 
hardware  and  software  for  science  and  engineering  applications  and  to  expand  the 
numbers  of  researchers  skilled  in  the  use  of  advanced  computing  technologies.  The 
NSF  Supercomputer  Centers  have  now  become  important  intellectual  centers,  driv- 
ing the  rapid  progress  of  scientific  computing. 

This  impact  is  summed  up  very  well  in  the  recent  report  of  the  Task  Force  on 
the  Future  of  the  NSF  Supercomputer  Centers  Program,  from  which  I  now  quote: 
"In  the  10  years  of  their  existence,  the  Centers  have  fostered  fundamental  advances 
in  our  understanding  of  science  and  engineering,  expanded  the  use  of  high-end  com- 
puting in  new  disciplines,  facilitated  the  major  paradigm  shift  of  the  acceptance  of 
computational  science  as  a  full  partner  in  the  scientific  method,  and  facilitated  the 
education  of  a  new  generation  of  computational  scientists  and  engineers  in  support 
of  that  shift." 

In  short,  this  program  has  been  a  noteworthy  success  by  several  different  meas- 
ures. 

Thorough  review  and  assessment  of  the  Supercomputer  Centers  program  after  10 
years  is  both  reasonable  and  expected.  NSF  has  instituted  two  major  external  re- 
views over  the  past  four  years,  providing  many  opportunities  for  the  affected  re- 
search community  to  express  their  views.  The  restructuring  plan  for  the  Centers 
program,  which  the  Subcommittee  will  hear  more  about  today,  has  resulted  from  the 
recommendations  of  the  Hayes  Task  Force,  and  I  am  pleased  that  Dr.  Hayes  is 
present  to  discuss  them. 

I  would  like  to  call  attention  to  one  particular  recommendation  of  the  Hayes  Task 
Force,  which  was  that  leading  edge  computer  centers  should  be  partnered  with  re- 
gional computer  centers  for  maximum  benefit  to  the  academic  research  community. 
We  have  such  a  regional  computer  center  in  my  congressional  district  in  Huntsville. 
I  would  like  to  submit  for  the  record  a  written  statement  from  the  Alabama 
Supercomputer  Authority,  which  indicates  some  of  the  activities  of  the  center  and 
indicates  how  regional  centers  can  contribute  to  the  new  NSF  Partnerships  pro- 
gram. 

In  light  of  the  past  performance  of  the  Supercomputer  Centers,  any  proposal  for 
restructuring  the  program  must  lay  down  a  convincing  set  of  arguments  that  change 
will  be  beneficial.  This  is  particularly  true  since  there  seems  to  be  agreement  that 
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a  principal  role  of  the  program  remains  unchanged:  To  provide  access  to  high-end 
computing  infrastructure  for  the  academic  science  and  engineering  community. 

I  look  forward  to  this  review  of  the  Supercomputer  Centers  program,  and  in  par- 
ticular, to  a  discussion  of  how  the  program  may  be  strengthened  and  how  the  pro- 
posed changes  will  affect  the  research  community,  which  has  come  to  rely  on  this 
important  resource  for  the  conduct  of  science  and  engineering  research. 


Alabama  Supercomputer  Authority 

The  Alabama  Supercomputer  Authority  is  pleased  to  submit  this  written  testi- 
mony to  the  U.S.  House  of  Representatives  Committee  on  Science  at  the  public 
hearing  March  19,  1996. 

Today's  budget  restrictions  dictate  that  Federal  agencies  must  conduct  their  busi- 
ness in  the  most  efficient,  effective,  and  economical  manner  possible.  In  many  cases, 
the  outsourcing  of  certain  support  functions  can  be  of  major  assistance  in  that  pur- 
suit. 

State  governments,  in  addition  to  private  firms,  often  can  provide  a  service  to  a 
Federal  agency  at  a  fraction  of  the  cost  which  would  be  required  for  the  Federal 
agency  to  acquire  the  corresponding  internal  capability.  One  case  in  point  deals  with 
high-performance  computing,  also  known  as  supercomputing. 

Several  of  the  fifty  states  have  followed  Alabama's  1987  lead  and  established  their 
own  research-oriented  supercomputer  centers.  Most  of  those  state  centers  partici- 
pate in  the  Coalition  of  Academic  Supercomputing  Centers  (CASC).  (Several  of  the 
states  and  Federal  agencies  have  also  followed  Alabama's  1989  efforts  and  intro- 
duced high-school  students  to  supercomputing  via  statewide  data  networks.) 

Supercomputers  and  their  necessary  infrastructures  and  support  personnel  are 
expensive.  When  a  Federal  agency  encounters  a  new  and/or  temporary  need  for 
supercomputer  services,  it  would  do  well  to  investigate  the  possibility  of  outsourcing 
those  services  from  an  existing  state  supercomputer  center.  Several  pilot  cases  al- 
ready exist;  the  Alabama  Supercomputer  Center  today  provides  services  to  the  U.S. 
Army  Missile  Command  and  to  the  Tennessee  Valley  Authority. 

Also,  the  National  Science  Foundation,  which  for  the  past  decade  has  funded  four 
Federal  research-oriented  supercomputer  centers,  is  recompeting  that  program  in 
1996  with  a  "partnership"  emphasis.  Several  of  the  state  centers,  including  Ala- 
bama's, will  be  entering  into  partnership-oriented  proposals  to  NSF  with  the  exist- 
ing Federal  centers. 


Alabama  Supercomputer  Authority 

The  concept  of  Federal-State  resource  sharing  can  probably  find  no  better  vehicle 
than  supercomputing  for  the  demonstration  of  the  concept's  merits. 

Accordingly,  the  Alabama  Supercomputer  Authority  recommends  that  the  House 
Science  Committee  consider  adoption  of  policy  statements  or  passage  of  legislation 
encouraging  collaborative  relationships  between  the  federal  research  agencies  and 
state-regional  high  performance  computing  centers.  Such  collaborative  relationships 
should  generate  substantial  cost  savings  by  reducing  the  need  for  these  agencies  to 
acquire  costly  technology  and  employ  highly  paid,  skilled  technicians  to  operate  gov- 
ernment-owned computer  centers. 

At  the  same  time,  such  collaborative  relationshipss  will  enhance  state  and  re- 
gional technological  advances,  and  foster  greater  use  and  understanding  of  high  per- 
formance computing  by  industry  and  by  faculty  and  graduate  students  in  the  na- 
tion's universities.  As  the  Members  of  the  Committee  will  recall,  these  points  are 
made  by  various  commissions  appointed  by  the  National  Science  Board,  beginning 
with  the  "Lax  Report"  published  in  1983  and  continuing  through  the  "Hayes  Report" 
published  in  1995. 

With  its  significant  experience  in  providing  "outsourced"  computing  assets  to  Fed- 
eral agencies  and  private  sector  companies,  the  Alabama  Supercomputer  Authority 
would  like  to  invite  the  Committee  to  host  a  field  hearing  later  in  the  year  at  our 
center  in  Huntsville.  The  Hearing  would  provide  Committee  Members  an  oppor- 
tunity to  review  outsourcing  and  collaborative  relationships,  how  such  arrangements 
have  strengthened  scientific  research  in  the  US  during  the  past  ten  years,  and  what 
steps  might  continue  and  strengthen  these  collaborations  in  the  future. 
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STATEMENT  OF  THE  HONORABLE  MIKE  DOYLE  (PA-18) 

SUBCOMMITTEE  ON  BASIC  RESEARCH 

Hearing  on  NSF  Supercomputing  Centers 

March  19,  1996 

I  want  to  thank  Chairman  Schiff  for  holding  today's  hearing  on  the  National 
Science  Foundation's  Supercomputing  Centers  Program.  As  a  member  who  is  fortu- 
nate enough  to  have  one  of  these  centers  in  my  District,  I  am  well  aware  of  the 
contribution  they  have  made,  and  will  continue  to  make  to  our  nation's  technological 
infrastructure. 

First,  I  would  like  to  welcome  our  witness  from  the  Pittsburgh  Center,  Dr.  Ralph 
Roskies.  He,  along  with  many  other  dedicated  individuals,  have  made  the  Pitts- 
burgh Supercomputing  Center  an  internationally-recognized  success.  The  PSC  has 
made  meaningful  contributions  in  many  areas,  notably  in  assisting  NIH  research, 
developing  weather  models  for  the  National  Weather  Service,  and  helping  EPA  with 
its  air  quality  assessments. 

Today's  hearing  examines  the  status  of  all  of  the  supercomputing  centers,  which 
I  believe  is  a  timely  and  worthwhile  undertaking.  We  have  before  us  three  relatively 
new  reports,  the  most  recent  of  which,  the  Hayes  Report,  calls  for  some  rather  sig- 
nificant action  to  be  taken  on  this  program.  I  am  hopeful  that  today's  hearing  will 
givee  us  an  opportunity  to  understandd  why  the  Hayes  Report  calls  for  a  departure 
from  what  has  been  an  unquestionably  successful  program. 

Again,  I  am  pleased  that  we  are  having  this  hearing,  and  hope  that  it  will  allow 
us  to  understand  the  significance  of  is  program  prior  to  our  acting  in  anyway  to  dis- 
mantle it. 


Testimony  of 

Ralph  Roskies  and  Michael  Levine, 

Pittsburgh  Supercomputing  Center 

before  the 

Subcommittee  on  Basic  Research 

OF  THE 

Committee  on  Science 

March  19,  1996 

Assessment  of  major  accomplishments  of  the  NSF  Supercomputing  Centers 

Program 

The  NSF  Supercomputing  Centers  Program  has  been  extremely  successful.  It  has 
helped  the  nation  retain  its  world  leadership  in  high  performance  computing  and 
communications.  It  has  enabled  very  significant  contributions  to  science  and  engi- 
neering; improved  the  state  of  the  art  in  computing  and  in  electronic  communica- 
tion; and  contributed  substantially  to  educating  the  next  generation  of  scientists  in 
the  computational  techniques  they  will  need  to  be  most  productive. 

The  talented  staff  at  the  Pittsburgh  Supercomputing  Center  (PSC)  and  our  affili- 
ated scientists  have  made  major  contributions  to  all  these  areas.  Many  of  our 
achievements  are  detailed  in  the  joint  submission  of  all  the  centers,  which  accom- 
panies this  testimony. 

The  Pittsburgh  Supercomputing  Center  has  particularly  championed  the  largest 
computations  feasible  with  today's  technology.  PSC's  resources  are  the  most  sought 
after  by  large  projects  nationally.  Last  year,  more  than  half  the  computing  power 
delivered  to  the  largest  projects  in  the  program  came  from  PSC.  We  have  enabled 
major  advances  in  science,  some  of  which  have  immediate  positive  impact  on  the 
quality  of  life  for  the  people  of  our  nation. 

For  example,  one  of  the  earliest  successes  of  the  Center's  program  was  work  done 
at  PSC  which  led  to  a  better  understanding  of  the  generation  of  smog  over  Los  An- 
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geles.  This  study  had  significant  impact  on  California's  regulations  to  improve  Los 
Angeles'  air  quality. 

You  will  hear  in  the  next  panel  from  Dr.  Kelvin  Droegemeier  from  Oklahoma. 
Working  together  with  us  on  the  world's  most  powerful  computers,  he  was  able  for 
the  first  time  to  make  accurate  predictions  of  the  course  of  severe  thunderstorms 
six  hours  ahead  of  time.  He  will  explain  how  this  will  lead,  among  other  things,  to 
safer,  more  convenient,  and  more  cost-effective  air  travel. 

Professors  Gregory  Olson  and  Arthur  Freeman  of  Northwestern  University  have 
used  our  facilities  to  design  ultra-high  strength  steel  for  weight-critical  applications 
like  naval  aircraft  landing  gear,  and  bearings  used  in  the  space  shuttle. 

During  the  summer  of  1995,  Professor  Steven  Taylor  of  Caltech  needed  the  full 
power  of  our  largest  supercomputer  to  model  the  Delta  II  launch  vehicle.  This  work 
was  done  under  severe  time  pressure  because  the  Air  Force  was  trying  to  under- 
stand a  malfunction  of  the  Delta  vehicle  while  launching  a  Korean  satellite.  With 
data  from  simulations  at  PSC,  design  engineers  confirmed  their  speculations  about 
the  source  of  the  problem,  and  this  led  to  a  successful  launch  in  November  1995. 

PSC  has  led  in  the  application  of  high-performance  computing  to  biomedicine  and 
healthcare.  We  were  the  first  national  supercomputing  center  to  get  extramural 
funding  from  the  National  Institutes  of  Health.  As  one  example  of  the  important 
work  done  in  healthcare-related  matters,  Professor  Charles  Peskin,  of  the  Courant 
Institute  in  New  York,  exploited  our  unique  capabilities  to  develop  a  very  accurate 
simulation  of  a  functioning  heart.  This  will  allow  better  design  of  artificial  heart 
valves,  requiring  less  experimentation  on  live  subjects. 

The  Pittsburgh  Supercomputing  Center  has  made  substantial  contributions  to  im- 
proving and  extending  the  reach  of  electronic  networks,  which  have  now  blossomed 
into  the  Internet  and  the  Information  Superhighway.  Because  of  our  networking  ex- 
pertise, we  were  able  to  design  the  infrastructure  for  Common  Knowl- 
edge:Pittsburgh,  one  of  the  earliest  and  most  highly  praised  pilot  projects  to  trans- 
form K-12  teaching  by  incorporating  computing  technology  into  schools.  The  work 
and  advice  of  the  Common  Knowledge  team  has  led  the  Pittsburgh  Public  Schools, 
a  major  urban  public  school  system,  to  undertake  a  5-year  $75M  plan  to  extend  this 
infrastructure  throughout  the  entire  school  system. 

PSC  and  its  users  have  won  many  distinguished  awards.  Probably  the  most  pres- 
tigious award  in  high  performance  computing  is  the  one  presented  at  the  major  an- 
nual U.S.  supercomputing  conference  "for  outstanding  contribution  in  the  applica- 
tion of  high  performance  computers  using  innovative  approaches."  PSC  users  won 
this  award  4  times  in  the  7  years  it  has  been  given — in  1989  for  the  air  pollution 
work,  in  1991  for  work  on  the  binding  of  proteins  to  DNA,  in  1994  for  the  heart 
modeling  and  in  1995  for  new  insights  into  turbulent  flows. 

PSC  and  Westinghouse  Electric  Corporation  won  the  ComputerWorld-Smithsonian 
award  in  Science  in  back-to-back  years,  1992-1993,  honoring  the  best  societal  uses 
of  information  technology  from  across  the  world.  Our  vital,  long-standing  relation- 
ship with  Westinghouse  has  been  an  excellent  example  of  the  synergy  between  gov- 
ernment, academia  and  industry.  We  were  able  to  get  a  very  fast  start  in  1986  be- 
cause Westinghouse  effectively  cloned  and  then  supported  their  existing 
supercomputing  infrastructure  for  the  PSC.  In  later  years,  we  were  able  to  propa- 
gate many  ideas  in  which  PSC  had  greater  expertise  to  Westinghouse  engineers. 
This  productive  give  and  take  continues. 

The  interaction  of  centers  and  their  users  with  supercomputer  vendors  has  been 
another  excellent  example  of  synergy  between  government,  academia  and  industry 
in  a  nationally  critical  technology.  Each  center  provided  valuable  feedback  to  the 
vendors,  and  improved  the  usability  of  their  machines.  In  return,  the  vendors  in- 
vested millions  of  dollars  in  the  centers,  which  has  gone  to  support  increased  capa- 
bility available  to  the  national  research  community.  We  have  worked  particularly 
closely  with  Cray  Research.  Some  of  our  developments  have  been  incorporated  into 
their  products.  We  took  very  early  machines  (PSC  got  the  very  first  Cray  Research 
massively  parallel  computer,  and  we  are  expecting  shipment  of  the  first  of  the  next- 
generation  machines  later  this  week).  Our  cooperative  efforts  have  given  the  na- 
tional research  community  very  early  access  to  the  newest  and  most  powerful  tech- 
nology. 

Developments  at  PSC  have  advanced  the  state  of  the  art  in  high-performance 
computing.  For  example,  we  did  the  earliest  experiments  linking  together 
supercomputers  of  radically  different  designs.  We  extended  to  the  supercomputing 
world  a  new  design  for  file  systems  originated  at  Carnegie  Mellon  University. 

Literally  thousands  of  students  have  taken  part  in  our  training  workshops,  and 
become  proficient  in  high-performance  computing  on  our  facilities  in  university 
courses  and  graduate  research. 
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Comments  on  the  new  NSF  recompetition  process 

It  is  very  gratifying  that  every  panel  investigating  the  supercomputing  centers 
has  found  them  to  be  a  great  success.  We  are  particularly  pleased  that  the  new 
recompetition  asserts  the  centrality  of  "the  most  powerful  computing"  to  the  na- 
tional scientific  enterprise.  The  emphasis  of  the  new  solicitation  on  partnerships  has 
led  us  to  a  fruitful  examination  of  new  approaches  which  will  be  embodied  in  our 
full  proposal.  We  are  confident  that  the  recompetition  process  is  both  open  and  fair. 

Our  major  regret  is  that  preparation  for  the  new  solicitation  has  undermined  co- 
operation between  the  centers.  That  cooperation  had  been  substantial,  taking  years 
to  develop,  and  was  serving  the  country  very  well.  There  was  no  lack  of  competition 
between  the  Centers  while  we  were  being  cooperative.  We  hope  that  the  former  level 
of  cooperation  will  be  restored  quickly  after  NSF  makes  its  next  decisions. 

The  general  apprehension  is  that  at  least  one  and  maybe  two  centers  will  be 
closed  as  a  result  of  the  recompetition.  We  do  not  believe  that  this  is  in  the  national 
interest.  The  program's  success  in  enabling  exciting  new  science,  as  well  as  educat- 
ing many  researchers  to  become  proficient  in  high-performance  computing,  has  in- 
creased the  demand  and  need  for  such  centers.  Each  center  has  made  unique  con- 
tributions to  the  program.  The  diversity  in  the  centers  program  will  be  compromised 
if  there  are  fewer  centers.  That  diversity  of  approach  has  greatly  benefited  the  na- 
tional effort.  The  program  has  made  this  a  very  exciting  time  in  computational 
science  and  engineering.  Now  is  not  the  time  to  be  drawing  back  from  ever  increas- 
ing success. 

Many  important  scientific  and  engineering  problems  require  considerably  more 
high  performance  computing  and  communications  capability  than  we  can  now  mus- 
ter. The  program  should  provide  the  resources  for  much  critical  new  work.  We  are 
convinced  that  the  rapid  pace  of  change  in  the  relevant  technology  will  be  sustained 
throughout  the  10-year  lifetime  of  the  new  initiative.  Successful  new  partnerships 
will  have  to  adapt  quickly  to  change,  as  the  NSF  centers  have  done  in  the  past  10 
years.  We  at  the  Pittsburgh  Supercomputing  Center  look  forward  to  the  opportunity 
for  making  substantial  contributions  to  the  flourishing  of  computational  science  and 
engineering  in  the  country  during  the  next  10  years. 


Answers  to  Questions  Submitted  to  Dr.  Paul  Young,  Assistant  Director  for 
CISE,  National  Science  Foundation,  by  Members  of  the  Subcommittee  on 
Basic  Research 

OTHER  R&D  AGENCY  ACTIVITIES 

QUESTION:  In  accordance  with  the  Hayes  report  recommendations,  has 
NSF  coordinated  with  other  R&D  agencies  to  develop  the  best,  and  most 
cost  effective,  means  for  advancing  scientific  computing  and  providing  ac- 
cess to  high-end  facilities  by  the  academic  research  community?  Are  other 
federal  agencies  developing  supercomputer  centers  with  leading  edge 
hardware  which  will  provide  access  for  academic  researchers?  Did  NSF  ex- 
plore a  multi-agency  program  to  meet  the  goals  of  the  new  Centers  Pro- 
gram? 

ANSWER:  Several  other  agencies,  notably  DOE,  DOD  and  NASA,  are  well  into 
development  of  leading  edge  computing  facilities.  In  general,  the  capabilities  and 
planned  uses  of  these  facilities  are  strongly  tied  to  the  missions  of  those  agencies. 
Academic  use  that  fits  the  general  mission  oriented  nature  is  allowed  at  some  of 
these  facilities,  but  general  use  for  the  academic  research  and  engineering  commu- 
nity is  not  available. 

NSF  and  the  Hayes  task  force  discussed  possible  multiagency  initiatives  and  such 
discussions  continue.  Federal  laboratories  were  explicitly  mentioned  as  potential 
partners  in  the  Partnerships  for  Advanced  Computational  Infrastructure  Program 
Solicitation  and,  at  the  present  time,  NSF  is  awaiting  the  proposals  to  see  what 
multiagency  activities  might  be  included. 

HIGH  CAPACITY  CONNECTIONS 

QUESTION:  The  restructured  Supercomputer  Centers  Program,  with  a 
main  site  and  satellite  sites,  will  require  high  capacity  network  connec- 
tions between  many  of  the  sites.  NSF  is  currently  funding  high  capacity 
connections  among  the  existing  Supercomputer  Centers  under  NSF's  Very 
High  Speed  Backbone  Network  Service.  Will  resources  for  such  computer 
connections  for  the  new  partnerships  come  from  the  new  program  awards 
or  from  other  NSF  programs  which  provide  for  high  performance  Internet 
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connections?  If  funding  will  come  from  other  NSF  networking  connection 
programs,  will  any  resources  be  available  for  high  capacity  network  con- 
nections in  support  of  research  projects  not  associated  with  the  Centers 
Program? 

ANSWER:  When  these  questions  have  arisen  from  potential  proposers  in  the 
Partnerships  for  Advanced  Computational  Infrastructure  (PACI)  Program,  we  have 
encouraged  proposers  both  to  look  at  the  networking  infrastructure  support  pro- 
grams of  the  Networking  and  Communications  Research  and  Infrastructure  (NCRI) 
Division  and  to  include  networking  support  in  their  PACI  proposals.  Our  intent  is 
to  work  closely  with  NCRI  to  optimize  the  effect  of  both  programs  on  high  speed 
networking  infrastructure,  not  only  for  the  PACI  sites,  but  also  for  the  entire 
science  and  engineering  research  community.  NCRI  resources  will  be  available  for 
meritorious  projects  not  associated  with  the  PACI  program. 

PACI  PROPOSAL  SUBMISSION 

QUESTION:  Since  the  main  focus  of  the  new  Supercomputer  Centers  Pro- 
gram will  continue  to  be  support  for  high-end  users,  why  do  you  allow  enti- 
ties other  than  the  leading-edge  sites  to  be  the  lead  institutions  submitting 
proposals? 

ANSWER:  We  only  require  that  each  partnership  have  at  least  one  leading  edge 
site.  We  felt  that  it  was  too  restrictive  to  require  that  the  leading  edge  site  also  be 
the  lead  institution. 

PACI  AND  HPCC  RELATIONSHIP 

QUESTION:  Explain  the  relationship  between  the  new  Supercomputer 
Centers  Program  and  NSF's  contribution  to  the  multi-agency  High  Per- 
formance Computing  and  Communications  Program. 

ANSWER:  As  documented  in  the  recent  National  Research  Council  report,  enti- 
tled "Evolving  the  High  Performance  Computing  and  Communications  Initiative  to 
Support  the  Nation's  Information  Infrastructure,"  NSF's  participation  in  HPCC  is 
both  broad  and  effective.  We  expect  that  the  Partnerships  for  Advanced  Computa- 
tional Infrastructure  Program  will  increase  NSF's  contribution  to  the  Basic  Re- 
search and  Human  Resources  component  of  HPCC,  and  also  to  the  Information  In- 
frastructure Technology  and  Applications  component,  while  still  maintaining  the 
major  NSF  impact  on  Advanced  Software  Technology  and  Algorithms. 

FUTURE  FUNDING  LEVELS 

QUESTION:  During  testimony  by  the  four  current  Center  directors,  the 
point  was  made  that  to  maintain  U.S.  preeminence  in  supercomputing  re- 
search, the  level  of  funding  required  would  be,  at  a  minimum,  this  year's 
level  adjusted  annually  for  inflation.  Please  comment  on  this  observation 
in  light  of  NSF's  requested  FY  97  budget  for  the  Supercomputer  Program, 
and  also  for  the  anticipated  flat  funding  for  the  RRA  account  for  FY  98  - 
FY  2002. 

ANSWER:  FY97  is  the  final  year  of  the  current  Centers  program.  Until  we  see 
the  new  proposals  we  will  not  know  how  much  pressure  they  will  put  on  the  FY98 
-  FY2002  budgets.  The  Center  Directors  were  probably  reacting  to  the  fact  that, 
with  additional  participants,  we  cannot  sustain  four  leading  edge  sites  and  partner- 
ships at  current  funding  levels.  However,  demands  for  computing  services  nave  al- 
ways exceeded  the  available  resources  with  the  funding  level  of  the  current  pro- 
gram. 

PACI  PROGRAM  SOLICITATION 

QUESTION:  One  goal  of  the  Partnership  program  is  to  encourage  a  full 
and  open  competition.  Of  the  four  existing  NSF  Centers,  one,  (SDSC)  is  cur- 
rently operated  through  a  cooperative  agreement  with  a  private  company. 
On  page  three  of  the  Program  Solicitation,  it  states  that  ".  .  .  it  is  expected 
that  proposals  will  come  from,  and  the  partnership's  cooperative  agree- 
ment will  be  with  a  U.S.  academic  institution."  This  statement  appears  to 
require  that  the  Center  currently  run  by  a  private  company  (SDSC)  cannot 
compete  for  a  future  award  without  undergoing  a  significant  restructuring 
not  required  of  the  three  other  Centers.  During  oral  testimony  by  the  Act- 
ing SDSC  Director,  it  appeared  that  NSF  had  resolved  any  concerns  relat- 
ing to  this  restriction. 
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ANSWER:  The  "....it  is  expected  that"  language,  was  included  in  the  solicitation 
to  reaffirm  that  the  principle  goal  of  the  program  is  support  of  academic  research. 
It  was  not  targeting  any  private  company  or  calling  for  a  major  restructuring  of  any 
current  arrangement.  General  Atomics  (GA),  who  currently  operates  SDSC,  was  in- 
formed of  this  the  first  time  they  asked  the  question  in  late  January  1996. 

(a)  Please  advise  the  Subcommittee  of  NSF's  interpretation  relating  to 
whether  a  private  company  may  compete  for  a  future  award. 

ANSWER:  NSF's  Grant  Proposal  Guide  states  that: 

"3.  For-profit  organizations:  U.S.  commercial  organizations,  especially  small  busi- 
nesses with  strong  capabilities  in  scientific  or  engineering  research  or  education. 
(See  Section  V.K.  for  specific  information  on  the  Small  Business  Innovation  Re- 
search (SBIR)  program.)  An  unsolicited  proposal  from  a  commercial  organiza- 
tion may  be  funded  when  the  project  is  of  special  concern  from  a  national 
point  of  view,  special  resources  are  available  for  the  work  or  the  proposed 
project  is  especially  meritorious.  NSF  is  interested  in  supporting  projects  that 
couple  industrial  research  resources  and  perspectives  with  those  of  universities. 
Therefore,  it  especially  welcomes  proposals  for  cooperative  projects  involving  both 
universities  and  the  private  sector. 

We  believe  that  General  Atomics  falls  within  this  guideline  and  will  continue  to 
do  so. 

(b)  The  Acting  SDSC  Director  expressed  concern  in  his  oral  testimony 
that  there  may  now  be  insufficient  time  for  a  private  company  to  put  to- 
gether an  adequate  proposal  by  April  15,  1996  under  the  new  NSF  interpre- 
tation. Please  comment  on  this  statement. 

ANSWER:  The  deadline  date  for  proposals  is  September  1,  1996.  The  April  15 
date  is  for  preproposals.  General  Atomics  (GA)  has  been  advised  that  the  intent  of 
the  preproposal  round  is  to  provide  feedback  to  proposers  and  was  not  intended  to 
be  used  as  a  screening  round.  We  will  accept  a  minimum  proposal  from  GA  or  any 
other  qualified  proposer  and  encourage  the  review  panel  to  provide  them  with  as 
much  feedback  as  possible 
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